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FOREWORD 



In general, junior colleges have not made a significant commitment to in- 
stitutional research. At the same time, like all other sectors of higher education, 
they are asked to look more introspectively at their operations at a time of 
critically scarce resources. Roueche and Boggs (3) assessed the level of com- 
mitment to institutional research jn junior colleges in their state-of-the-art 
monograph. Earlier, Swanson (5) had provided a comprehensive assessment of 
institutional research in junior colleges across the country. Both reports found 
that the activity was narrow in scope and that only a fraction of the colleges 
could report actual on-going institutional investigations. Earlier reports appear 
to bear this out 

More recently, Schafer and Turner (4) presented a topical paper to the Florida 
Community Junior College Inter-Institutional Research Council, outlining an 
analytical framework for viewing institutional research in the junior college. 
Their introductory comments deserve restating here: 

The issue of research has only recently become prominent in the com- 
munity college environment. Pioneer writers and publicists of public junior 
college development, in concentrating largely on the teaching function of 
the instf" itlon, gave but passing attention to the activity. Indeed, these early 
advocates often differentiated the community college from senior in- 
stitutions in terms of its teaching emphasis and quality as contrasted to the 
lighter stress on research, An orientation away from research thus became 
an inherent part of the community college mystique, and was usually tran- 
slated by it into heavier teaching loads and more faculty involvement with 
students. 

The last few years, however, have seen a growing inquiry into whether com- 
munity colleges are, in fact, fulfilling the high aims once so confidently an- 
nounced. Though still muted, demands for evidence are beginning to be 
heard. Even the most ardent proponents of the movement appear to 
recognize shortcomings and a consequent need for a firmer base for in- 
stitutional philosophy and methods [that] would come from empirical study. 
Yet there Is no general agreement on the scope of desirable research, or on 
preferable approaches for its accomplishment. Not only is there a lack of 
consensus, but discussion of the subject suffers from semantic obscurities 
and scanty information. The role of research in the institution does not 
seem to have been subjected to much rigorous analysis. 
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So many such statements substantiate the absence of research that we need 
not belabor the point. We must emphasize that, in an era of scarce resources, 
when a wide range of publics are demanding better analysis of institutional 
goals and more sharply defined criteria to test the return on the investment of 
educational dollars, we as junior colleges find ourselves without a well planned 
program of institutional self-study. When institutional research Is conducted, 
the data are often poorly related to decision making or fall into the category 
called "fugitive data” that never reaches the hands of those who might be able 
to use it. In summary, the bibliography on institutional research in the junior 
college bears out the same conclusion — little or none is being carried out by 
most of our institutions. 

Recently several junior college Institutional research officers met in St, Louis to 
develop a National Demonstration project in Institutional Research under the 
auspices of the League for Innovation in the Junior College. Representatives 
from Dallas, St. Louts, Peralta, the Coast, Cuyahoga, Santa Fe, and several 
other districts met to determine how to implement a model demonstration 
project on a pilot basis in six to eight major districts across the country. 

They agreed that five major components should be included in any comprehen- 
sive program, and recommended institutional research that: 

T assesses the allocation of resources 

2. assesses student potential 

3. assesses achievement 

4. analyzes curriculum and program needs and priorities 
5 a assesses the college’s impact on the community* 

A sixth category, which Is not reaily a separate component, was added, namely, 
providing such research assistance as advising a staff member on research 
design, dissemination of reports, and other technical or consultative services. 

Most of these categories are seif-explanatory, but some sub-categories need to 
be explained. In effect, the St, Louis group developed a taxonomy of in- 
stitutional research functions under the five broad categories listed above. 

As an example, component one (1) was further developed in this manner, 

1. Institutional research that assesses allocation of resources 

Basically, this first component would attempt to develop methodologies 
and data gathering that permit better management decisions. Three basic 
considerations are subsumed under this component. The first would in- 



valve [how] institutions allocate fiscal resources; the second [how they] 
allocate personnel resources; and the third. . .would systematically analyze 
[how they] allocate physical resources. Included under the analysis of 
fisca; resources would be such. . studies as cost effectiveness* allocation 
of resources for specific functions in the institution* cost benefit analysis, 
and unit cost analysis relating to. . .plant operation, maintenance, cost per 
Instruction* etc, 

Each component has been developed so that a handbook or manual can be 
written on how to set up various institutional research services. 

Component three (3), for example, relates achievement to four sub-categories; 
(a) to Incoming student characteristics, (b) to instructional methodologies, (c) to 
student expectations, and (d) to institutional climates. 

In general the League’s National Demonstration project would attempt to renew 
an interest in institutional research and to implement model demonstration 
programs in several community colleges. 
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REVIEW 

The days of flying by the seat of the pants are over— -In both aviation and 
higher education. Early in the 1960s, in a three-year Carnegie-funded study, 
Pourke and Brooks observed that, as booming enrollments force the costs of 
higher education constantly upward, more legislators and executive officials 
will ask college and university officials for more refined data about their 
operations {4:64-65). 



As if in fulfillment of this prediction, one view became clear in the 1970 meeting 
of the Education Commission of the States: 

Many governors and top state education officials believe that the public is 
becoming more and more unhappy with the nation's educational system 
and that education must find ways of measuring how effective it really is 
(5:1-2). 



Institutional research emerged in the 1960s as a form of organizational self- 
study to help colleges and universities gather information about their internal 
operations and how effectively they use their resources. Officers hope to use 
these data for making informed judgments rather than guessing or relying on 
intuition (4:44). 

The need for institutional research among colleges and universities applies 
equally to the nation’s junior colleges — perhaps even more so. The need could 
be based on the open-door admissions policy and the concomitant challenge to 
counsel wisely, place students and follow up on them; on the acceptance of oc- 
cupational education as a proper function of the junior college with its accom- 
panying obligation to survey employment trends accurately and to maintain 
close liaison with business and industry; on the need to maintain such stan= 
dards of instruction as will ensure satisfactory transfer of students to senior in- 
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stitutions; and on the need to keep the local community informed and willing to 
support the college— their college. This partial list presents a strong rationale 
for junior colleges to engage in internal research. 

The response from junior colleges, however, has been extremely limited in both 
quantity and quality. Surveys conducted between 1959 and 1968 indicated that 
less than one-quarter of the institutions undertook any coordinated research 
activity (6: 11), and that less than one-fifth had formal programs of institutional 
research (7:180), Fewer than 2% had written policies; fewer than 10% had a 
separate budget item for research (7:182), The average junior college 
completed only one institutional research study per year, nearly half of them 
devoted to ‘‘students" (3:47), Six studies out of seven were characterized by 
faulty design, poor methodology, and inappropriate conclusions (3:36). 

Institutional evaluation is ultimately the president’s responsibility. Aaron Brum- 
baugh, in his pioneering monograph, first noted that the key to effective 
research Is the ability of the college president and his staff to ask the right 
questions (1:2). Nearly ten years later, Roueche and Boggs concluded that, If 
the president is asking the right questions, the odds are good that the answers 
will be found (3:50). 

Where does the president turn for help In asking the right questions? 
Specifically, where does he turn for help in assessing his college’s present ef- 
forts in institutional research or for help in reviewing the policies and 
procedures behind his institutional research operations? Assuming that a for- 
mal organization of effort is justified, where does he turn for help in identifying 
a qualified director/coordinator and in structuring a viable position? 



ERIC 



A review of the literature, oriented principally to senior colleges and univer- 
sities, is devoted largely to the need for and the product of , not the process of , 
Institutional research. Findings of the 1969 Van Istendal project covered in- 
stitutional responses that summarized the need for planning, organizing, and 
establishing procedures for a program of institutional research (8:82), 

RECENT RESEARCH 

A recently completed study by Cook (2) was an exploratory, empirical work 
drawing on the experience of many knowledgeable people. Its purpose was to 
develop guidelines for junior college administrators to use in organizing, ad= 
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ministering, and evaluating their programs of institutional research. The 
following paragraphs summarize the study and its findings. 

The initial data for the Cook study came from a review of the literature. Each 
relevant concept Identified was treated as a position statement and phrased as 
a positive criterion. One hundred ten suggested criterion statements were 
prepared in checklist form for a mailed questionnaire. 

Institutional research concerns not only those who prepare the data but also a 
variety of consumers interested in the findings. The jury of 215 members who 
validated the 110 suggested criterion statements was therefore selected from 
five populations: public junior collage presidents; junior college directors of in** 
stitutional research; professors of higher education with special experience in 
either the junior college or institutional research; officers of the regional ac- 
crediting associations working with junior colleges; and state-level ad- 
ministrators of junior college education. While breadth of perspective was thus 
encouraged, the study’s purpose was still to develop practical guidelines for 
junior college administrators. Accordingly, the greatest voice was given to 
junior college presidents and directors of institutional research. 

Replies from 199 of the 215 jurors showed a median of three years’ experience 
in their specialty at their place of employment In addition, 155 jurors reported a 
median of four years’ experience at previous institutions. The mode was two 
years and three years respectively. 

The participants from the junior colleges represented institutions of all sizes. 
Responses from 138 junior college jurors placed the median enrollment in their 
institutions at 1,358 full-time students; responses from 134 jurors placed their 
median total enrollment at 2,690 students. 

EMERGING GUIDELINES 

By its nature, the focus of the Cook study was not on prevailing practice, but on 
what “should be. H The conclusions drawn are, by their substance, guidelines 
recommended for organizing, administering, and evaluating programs of in- 
stitutional research in junior colleges. These conclusions are paraphrased as 
follows: 

1, The questionnaire response rate and comments volunteered by respon- 
dents show a considerable interest in institutional research and recognize 
ths need for guidelines to organize and administer it. 
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2. Although many topics are appropriate for Institutional research in a junior 
college, assignment of priorities permits a better concentration of effort, 

3. Highest priority is given to the “student'’ category of inquiry. Including 
enrollment trends, student profiles, attrition, and follow-up studies, 

4. Inquiry Into institutional goals also has a high priority, as do the relation- 
ship of goals with ‘‘community” or institutional environment and the 
adequacy of statements of purpose is a reference for decisions, 

5. In curriculum and instruction, prime importance Is attached to the 
relationship of curriculum structure to goals, the relevance of course of- 
ferings to the nature of the student body, and the conditions that en- 
courage experimentation with new techniques of teaching and learning. 

6. High priority in management-related inquiry is given to space-use studies, 
budget analyses, funding trends, and improvement of relations with area 
high schools* 



7, The office of institutional research, while it should collect data needed for 
direct assistance in development, evaluation, and control of operations, 
should remain free of Identification with any specific policy or action, 

8. The office should be guided by a philosophy that Invites individuals and 
offices to engage in certain aspects of the total task. A coordinating com- 
mittee of all officers so engaged should be established. 

9 A General advisory committee should adopt a statement of basic prin- 
ciples for institutional research activities of the college, advise on project 
priorities, and assist in planning activities. 

10. In studies of academic matters, the office should provide design and 
evaluation services to individual faculty members who are experimenting 
with new approaches to teaching and learning. It should also summarize 
and disseminate experimental findings from other campuses. 

11. The office should maintain a master list and file copies of all institutional 
studies (completed and in progress) and of selected publications related 
to present or contemplated college policies and programs. 
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12. In addition to the findings, institutional research reports should include 
the population sti died, the data-gathering techniques, interpretational 
cautions, alternate courses of action, and the implications of each, 

13. The director ot institutional research should not only initiate studies, but 
also repeat certain ones over a period of time to perfect methodology and 
identify trends. He should also sharpen definitions and develop 
procedural standards for greater consistency in the gathering and main- 
taining of data by a!! units, 

14. He should participate in Inter-institutional research by completing and 
returning questionnaires seeking institutional data from him, and by 
studying practices at other colleges, 

15. The circulation of the research reports Is best determined by the purpose 
and content of the study, whether or not this practice Is specified in the 
statement of policy. 

16. The director should hold a staff position and report to the president. 

17. He should be a member of the administrative councils, standing and/or 
special study committees, and a consultant to offices responsible for ob- 
taining financial support, 

18. In the course of his work, the director should take part in the decision- 
making process, though actual decisions must be made by line officers 
with operating responsibilities. His role should be one of interpreting data, 
identifying trends, and suggesting alternate courses of action. 

19 . in the smaller junior college (those having less than 1,000 full-time 
students), the office of institutional research should be coordinated by an 
administrator on a part-time basis. In the larger institutions, he should be 
full-time. 

20. The qualifications most desired in the director are graduate work In 
methods of educational research and statistics, and knowledge of 
systems analysis and electronic data processing. Acquaintance with com- 
munity college operations and graduate work in higher education are also 
recommended. 
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21, The office should be adequately funded by a separate item in the college 
budget, for operating expenses, for in-service training, and for the ser- 
vices of consultants for specialized studies. 

22, As a cautionary note: inquiry into some of an institution’s internal 
operations is hazardous, and the office of institutional research is well ad- 
vised to wait for an invitation to work in such areas. Examples are: studies 
of faculty, their morale, and productivity; incorporation of new knowledge 
into, and elimination of duplication from, the curriculum; preparation of 
educational specifications and priorities for construction; administrative 
functions, policies, and procedures, and their acceptance by the faculty; 
the views of legislators toward the college and prospects for increased 
funding; and public understanding and support of community services. 

23, As a further caution: while the office of institutional ^search should avoid 
becoming separated from the study of immediate proMems, it should not 
become involved In the actual implementation of ^ orations. To perfect 
methodology and identify trend data, the office should not relinquish its 
jurisdiction/involvement with studies that must be repeated annually or 
periodically. To avoid dissipating the strength and/or focus of its efforts, it 
should not become the secretariat for faculty committees or become in- 
volved in the conduct of faculty workshops. In preparing his reports, the 
director should not attempt to escape his responsibility for interpreting 
data and developing implications by merely letting 'the facts speak for 
themselves.” Finally, h© should not attempt to govern the circulation of in- 
stitutional research reports. 

CONCLUSION 

Conclusions from the total study, including the questionnaire survey, juror com- 
ments, and the review of the literature, lead to this observation: The essential 
first step Is an administrative stylo characterized by reliance on data gathering 
and information as requisites for decision making — supplementing, if not 
replacing, intuition and tradition. The president is the key to setting this style. 
The administration and faculty, but particularly the president, must display a 
willingness to use data and act on them. Without a prior commitment to have its 
findings put into practice, research is doomed to failure. Conversely, if the 
junior college president is committed to institutional research, the activity will 
be supported and probably succeed. 
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INSTITUTIONAL RESEARCH AT THE 
TWO-YEAR COLLEGE: 
BUILDING THE FOUNDATIONS 



FRED A, SNYDER 

Director, Research and Planning Division 
Virginia Department of Community Colleges 

Early in fils efforts to begin a program of institutional research, the would-be 
researcher may be frustrated by the potential scope of his role. If he is an ex- 
perienced two-year college educator, he may think about changing educational 
objectives, "new" students and implications for the educational program, con- 
flicting rationales for learning and grading, and emerging instructional systems 
with emphasis on '-mastery'* of learning. He may be reminded that the com- 
munity college must remain aware of the community and its cultural, economic, 
and manpower aspects. He can expect the president to want studies on room 
use, curricular cost analyses, faculty loads and costs, and other management- 
oriented projects, The president may go so far as to note the need for cost- 
benefit alternatives for basic educational-management decisions that the 
college must face in the next several years. 

The press for accountability has descended on the two-year college and cannot 
be put off. Legislators, board members, taxpayers, even students and parents 
are asking the college to account for its efforts and resources. It must respond, 
for the alternative to self-examination and direction is submission to interests 
beyond the college. These conditions have given crucial impetus to the role of 
the institutional researcher. 



This paper is addressed to the researcher-aspirant , who is just beginning to 
plan and conduct institutional research. It provides practical advice on a num- 
ber of issues: 

Laying the groundwork 
Developing priorities 
Sustaining Institutional support 
Involving the faculty 
Assessing results 
Maintaining professional grov/th 
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LAYING THE GROUNDWORK 

Whether the role is fuil-tlrno or part-time, the sine qua non of the aspiring 
researcher is to have the role confirmed by the president or another ad- 
ministrator to whom he must turn for support. Institutional acknowledgment of 
the research role arid knowing the line? of authority and responsibility are 
essential jf research is to get done, Th® full-time researcher will usually have lit- 
tle problem with role recognition and supervisory relationships, but if his 
previous job was in the same institution, he must make sure to whom he is now 
responsible. It is not practical here to indicate to whom he should report, as the 
scope of his roie should determine that. If the role is restricted to study of 
student characteristics and outcomes, he may well report to the dean of 
students; if to instructional systems development, he should report to the dean 
of instruction; if to facility utilization, costs, and related management studies, he 
may report to the dean of administration. 

Where a comprehensive program of institutional research is intended, the direc- 
tor should report to the president, or, in larger colleges, to the vice-president for 
planning and development. The advantage of reporting to the chief executive 
officer is the likelihood of greater understanding and support. 

The part-time researcher may find several organizationa 1 relationships. He may 
continue to report to his earlier supervisor— e.g,, director of counseling, dean of 
students, or academic division chairman=for both aspects of his dual role. The 
part-time researcher may report separately for each role to two supervisors— for 
example, to the director of counseling and to the dean of students for his 
respective counseling and research roles. Some conflict may result from this 
dual supervisory approach. Of course, if the research role is expected to 
predominate, his relationship should be with his (R supervisor. Where a college 
can support only a part-time role, the writer favors the latter arrangement. 

As many two-year colleges cannot support a comprehensive research effort, at 
least during their early years, both the scope and the early research objectives 
must be clearly specified, in one college, the first year of !R emphasized 
developing a data base and descriptive information about students and 
educational outcomes. The second year was on special student groups, the 
community, and the faculty. The third year continued this diversity, but em- 
phasized instructional research. Since resources ore always limited, however, 
decisions must be mad© at the beginning on what research will (and will not) be 
done. 
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Available resources are ideally tailored to the scope and extent of activities to 
be conducted. If the proposal for the new research function was well- 
developed, it will have provided for clerical help, office space, travel and related 
expense, and data processing. It will also have included a total budget, to be 
administered by the director of institutional research. 



In many instances, however, institutional research will emerge In a less 
auspicious manner. An existing staff member, perhaps granted released time 
from current duties, may find no definite support, no plan for clerical aid, work 
epace, supplies, travel expenses, or data processing. He must remember to plan 
for better support during the next budget year. Such planning is discussed in 
subsequent sections of this paper. 



The researcher may not know how to assess needed resources before his ob- 
jectives and duties are clarified. His supervisor may already have determined 
these, or he may merely remark that “we need more information about our 
students and our graduates. . In any case, clear objectives must be set for 
planning the first year’s activities and be further developed into a list of 
research projects. Such planning, Including specific, approved projects, Ss 
likely to elicit better support than If It Is called just "research.” It also allows for 
a rational adjustment of the scope of approved research projects to existing 
college support, thus increasing the probability of achieving early research ob- 
jectives and gaining solid support for subsequent activity. Such early planning 
will allow the researcher to: 



1, focus only on the project or projects of greatest acknowledged value to 
the college 

2. avoid cost overruns and the resulting inability to complete projects 
because of lack of funds 

3. demonstrate a model of rational planning and acquire experience for sub- 
sequent planning 

4, demonstrate the value and practicality of institutional research, even at a 
small college. 
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DEVELOPING PRIORITIES 

The problem of developing priorities for IR activities has several aspects, two of 
which are the secondary activities often attached to the research office and the 
need to choose the most promising research projects from the many proposed. 

At the two-year college, the function of institutional research is often accom- 
panied by a variety of data collecting and reporting activities. The researcher 
may be expected to coordinate an array of periodic reports to state and federal 
agencies. With adequate clerical support, this function can usually be absor 
bed, but it tends to displace the more significant and valuable research func 
tion. The danger may be more pronounced with the part-time researcher, and 
the writer suspects that many find little time to do their work because of this 
largely routine data collection and reporting. Adding the role of handyman or 
assistant to the president to the researcher’s activities creates a similar 
problem. 

Clearly such activities as providing miscellaneous services for the president 
may be necessary, but they must be placed in proper perspective and not 
allowed to displace the more valuable research activities. If communications 
between supervisor and researcher are reasonably easy, the matter can be ad- 
justed. 

The crucial problem Is choosing the most necessary research projects from all 
that might #be done. A reasonably creative educational researcher can see a 
score of potentially valuable projects, but limited time and resources prohibit 
them. The development of research priorities should be tied to the educational 
objectives of the college. For example, if a college wants to enlarge educational 
opportunities for all socio-economic and cultural strata, the following sequence 
of research objectives might emerge: 

T describe the characteristics of the current student body 

2. examine the retention and graduation patterns of recent students 

3. study the demographic and employment characteristics of the community 

4. assess the views of specific population groups on the college and its 
programs. 
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The results of this sequence can be used to modify admissions procedures, 
educational programs, student services, and public information, and can con- 
tribute to such far-reaching decisions as where to locate new campuses and 
what new programs to develop. Clearly, only a large IR effort could carry out all 
the studies simultaneously, but a smaller effort could accommodate them over a 
longer period. Finally, research priorities should not be inscribed in gold, but 
must respond to new conditions at the college. 

SUSTAINING INSTITUTIONAL SUPPORT 

The president will be keenly interested in activities that will support the 
educational objectives of the college, and, as institutional research demon- 
strates its vitality and usefulness, his support can be expected to grow. His sup- 
port is essential to the whole institutional research effort, which must be dir©Gt- 
ed toward objectives he approves. The researcher must also work toward useful 
"products/' namely, well-written research reports, lists of recommendations (if 
he is expected to develop them), and a definite procedure for disseminating 
research findings, 

A common impetus to starting a program of institutional research is the need to 
conduct a college self-study or to prepare for an accreditation visit, for 
preparations must begin well in advance. If they begin early enough, the re- 
searcher can make a considerable contribution. Administrators and faculty 
alike will come to acknowledge the value of a continuing research program. 

If the research program is to be an on-going activity, it must be upheld by 
adequate budgetary planning. Personnel costs will vary by qualifications of the 
personnel and by geographic region. The use of a multiple-register calculator is 
essential for tabulations; It is not a substitute for free access to a computer. If 
programing assistance is not available from the college computing center and it 
must be contracted for, the budget must be increased accordingly. Funds are 
necessary for routine travel in the region of the college, plus two trips annually 
to national association conferences, if th© researcher is to keep up with new IR 
activity. 

INVOLVING THE FACULTY 

Institutional research is generally considered valuable only if findings are 
disseminated to those who conduct the educational processes and are incor- 
porated in subsequent practices. The users of research are the faculty and ad- 
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ministrative staff. The not-so-old adage that community colleges are teaching 
institutions (and therefore their faculty should eschew research and 
publication) has had an unfortunate effect. It has contributed to a static model 
of traditional instruction and curriculum design, usually modeled on the four- 
year college or university. To the extent that the community college is commit- 
ted to educational programs and services relevant to a specific community, to 
“new" student groups, and to new educational programs, the emphasis on 
“teaching only” has been counter-productive. The faculty at the community 
college should also engage in innovation and in research activities on 
educational programs, students, and learning outcomes. 

Both the relevancy and use of research are enhanced by instructional and ad- 
minstrative staff participation, which appears to be increasing. As a result, 
several principles are offered as a guide to obtaining faculty support. 

1, staff participation should occur in their primary areas. For example, the 
mathematics instructor might be encouraged to conduct a study on 
student attitudes toward mathematics, and use the results In subsequently 
developing a way to teach beginning mathematics students. 

2, Specific research and development projects should be officially 
acknowledged by the college. This can be don© either through released 
time from teaching or other duties or through additional compensation- 

3, Faculty involvement in a project should be preceded by a contract signed 
by the faculty member and a college representative. It should describe the 
project objectives, conditions of time released or compensation granted, 
and finished product (written report, instructional system, instructional 
devices, etc.). Such a contract must be followed in spirit rather than 
literally, as early plans will often be inadequate. The institutional re- 
searcher should review and approve such agreements, fo/ he can do 
much to assure their efficacy. 

A faculty committee for institutional research has much in its favor. It can help 
personnel identify research needs, assess the efforts, encourage faculty in* 
volvement as appropriate, and generally help to relate research efforts to the 
basic functions of the college. The nature and rol© of the committee cannot be 
generalized completely, as they relate to several local conditions, A research 
committee may contribute effectively to a broad and comprehensive program, 
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but relatively little to be a narrowly defined or part-time program. Generally, the 
committee members should have soma appreciation for institutional research 
and how It can enhance the operation of the college. The committee's role 
should be expressed in writing, and be subject to occasional modification. 
Research committees have several appropriate functions; 

1. assist the director In identifying research needs 

2. serve as liaison between the research function and the faculty as a whole, 
communicate the purposes of research, consider techniques for 
disseminating the findings, and relate current or recent research findings 
to current problems faced by faculty groups 

3. assess the value of past research activities and recommend new direc- 
tions. 

ASSESSING RESULTS 

The assessment of the results of his activities may be far from the mind of the 
beginning researcher, but, after starting several specific projects, the need for it 
will become apparent. Annual budgeting requires that research activities (even 
objectives) be specified according to personnel, equipment, supplies, and 
related expenses. How dollars are allocated to the research function depends 
on the perceived value of past and future outcomes. Several criteria for 
evaluating the IR program are: 

1, Were research projects completed and results reported to the president, 
administrators, and other groups? 

2, Were the objectives of the projects realized? Were the findings expressed 
in useful form for other college educators? 

3 S Were the coats of the projects in line with anticipated costs and in line 
with costs at other colleges? 

4. Do Instructors and administrators have a positive attitude toward the m= 
stitutional research effort? 

5. How have the results of past research been incorporated in the operation 
of the college? 
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6 What recommendations exist for future research projects and 
procedures? 

The basic responsibility for assessing the results of research activity should lie 
with the director. He has direct access to the data, and his assessment is con- 
sistent with self-study by an academic division or by the college. A prime value 
of self=study is the knowledge gained by the assessor, but the research commit- 
tee should probably share in the assessment. At any rate, a written report of the 
evaluation should be submitted to the president or the director’s superior ad- 
ministrator. 

A formal review of research activity should not be made before the end of the 
second year of operation, as the payoff from most research activity will not 
begin during the first year. 

MAINTAINING PROFESSIONAL GROWTH 

An accepted premise in higher education is that staff members must work in an 
environment that nurtures their professional growth, regardless of professional 
role. This applies especially to the new researcher, who may have to work not 
only without researcher colleagues but with limited experience himself. 

Directions for professional growth vary among individuals. One director may 
need to sharpen his knowledge of statistical analysis. Another may have a rich 
background in data analysis, but less in the philosophy and role of the com- 
munity college. Still another may have experience in social and community 
research, but little in education or learning. Many will lack experience in how to 
start a new research program. While there is no easy fulfillment of such needs, 
the new researcher will want to avail himself of ail chances to enrich his ex- 
perience. They include reading professional publications, visiting advanced 
research facilities, and attending professional conferences. 

It is not practical to include a comprehensive list of periodicals in this paper, 
but several are noted to Indicate the breadth of possibilities: 

Junior College Research Review provides occasional summaries of 
research on selected topics. Perusual of back issues through 1968 Is 
recommended. 

College Student Personnel Journal illustrates research on topics related 
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to students and student services. Many other journals In counseling, 
education, psychology, and sociology would be helpful. 

Research in Education abstracts monin ly all studies and reports submit- 
ted to the Educational Resources Information Center. All items related to 
the two-year college are listed under the heading ‘‘Junior College." 

Junior College Journal reflects a basic two-year college philosophy and 
contains some research articles. A perusua! of the last five years issues is 
recommended. 

Chronicle of Higher Education, a weekly, is the only authoritative national 
news report of hicjner education and presents an overview of current 
issues. 



\ 

I 

$ 




Actually, no list of publications could be complete, as the interests of resear- 
chers are wide and the literature profuse, A growing number on research and 
on the two-year college are being produced by AAJC, ERIC clearinghouses, 
research canters and consortia, and individual colleges. A recent summary of 
publications on higher education may be useful to those interested in the ad- 
ministration of higher education (2), 

Of immediate value to the new institutional researcher is the monograph Junior 
Co I lege Institutional Research: The State of the Art (3). It reviews institutional 
research practices at selected two-year colleges and presents several con- 
clusions and implications. A useful bibliography is Included, even though it is 
now three years out of date. For the more sophisticated researcher, the book In- 
stitutional Research in the University (1) discusses the broad Institutional 
research function. Separate chapters treat specific aspects of IR, so that the 
reader can concentrate on his concern of the moment. The serious researcher 
should not avoid this book merely because it has a university setting; many 
chapters apply to the two-year college. 

The importance of field trips to centers of research cannot be overestimated. 
They can illustrate the how-to-do and what -to-do, and allow one to meet ex- 
perienced and talented researchers who can give advice and assistance. Many 
two-year colleges with existing and useful research activities are amenable to 
receiving visitors. 
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Professional conferences continue to be a prime source of personal enrichment 
and renewal. Both the American Educational Research Association (AERA) and 
the Association of Institutional Research (AIR) have active two-vea r college 
subgroups, making attendance at their national conferences worthwhile. Other 
local and state research groups abound, but they vary in pu pose and relevance 
to the two-year college. Information abou* them can usual!/ be obtained from a 
neighboring college, 

The above activities are no substitute for conducting research and writing 
reports. Direct experience, combined with visits, confero ices, and workshops, 
will accelerate professional growth to the mutual benefit of the individual and 
the college. The research budget should allow for personal travel expenses, 
and good judgment should insure a proper relationship between research ac- 
tivities and travel, 



1 . Dressel, Paul L. and Associates, Institutional Research in the University: A 

Handbook. San Francisco: Jossey-Bass, lnc. s 1971. 

2. Hefferlm, Lon, and Phillips, Ellis L* Jr. information Services for Academic 

Administration . San Francisco: Jossey-Bass, Inc., 1971* , 

3. Roueche, John E,, and Boggs, John R. Junior College institutional 

Research: The State of the Art. Monograph no. 2 Washington, D.C.: 
American Association of Junior Colleges, 1968. 
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STANDARDS FOR STATISTICAL SURVEYS* 



Introduction 

Recommendations for planning and conducting statistical surveys have been 
developed by the Bureau of the Budget in consultation with the statistical agen- 
cies of the Federal Government, They are intended as guides to good statistical 
practice in surveys conducted by federal agencies or under federal sponsor- 
ship, by universities, market research firms, trade associations, or other 
organizations. 

As used here, a “statistical survey” is one intended to supply data on a par- 
ticular class, segment, activity, or area, but on which no administrative action is 
taken on the basis of the individual return itself. 

Standards cannot be applied uniformly or precisely in all situations. Special 
considerations may be involved in exploratory, experimental, or methodological 
surveys, or in pilot or preliminary surveys to evaluate alternative approaches or 
techniques. Similarly, no single survey design can meet all requirements. A par- 
ticular design should be appropriate for a given set of purposes and consider 
time and cost. Such consideration will help to determine, for example, whether 
to conduct a survey by mail or by personal interview, whether to use complete 
enumeration, some systematic sample design or “cut off” procedure, or some 
other device for selection of the respondents. 



Survey Standards 



The first step in planning a survey is determining the precise purposes to be 
served by the information to be collected, by the hypotheses to be tested, or by 
the problems to be solved, Prepare a concise statement of the quantitative infor- 
mation needed and the exact purpose it will serve. 

Before the scope and content of a survey are determined, prior and current 
work In the field should be reviewed, and the problem defined in relation to it. 



* Executive Office of the President. U.S, Bureau of the Budget, Division of Statistical Stan, 
dards, March 28, 1952, Circular No. A 46. 
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Decisions should be reached on each of the following points according »o the 
purposes, costs, time, degree of precision, administrative limitations 
(established practices, location of field offices, etc.), snd need for including or 
excluding specific areas; 

1. Respondents. Define the respondent group by the classes, segments, or 
areas to be Included, and the availability of source materials, such as mailing 
lists, maps, and directories. 

2. Extent of Coverage . Within the respondent group selected, decide 
whether enumeration should be complete or partial; if partial, whether by means 
of a probability sample, “cut-off’ point, or other plan. (The sampling plan for 
partial-coverage surveys is described later on.) 

3. Frequency and Timing. Determine whether the survey is to be single-time, 
recurrent, or periodic, and, if periodic, at what interval, Consider the timing in 
relation to cyclical or other variations over time in the data being studied, 

4. Method of Collection. Decide if the information Is to be collected by mail, 
personal interview, telephone, telegraph, etc., or by some combination of them, 

5. Consideration of Nonsampling Errors . Before the final decision on 
method, Identify the main sources of bias and of accidental and nonrandom 
errors anticipated from alternative procedures, make plans for testing the 
reliability of the data and studying errors of response and other nonsampling 
errors, including interviewer bias. 

6. Standard Definitions and Classifications, Wherever possible use the stan- 
dard definitions and classifications issued by the Bureau of the Budget so that 
the resulting data will be consistent with those from other projects. They in- 
clude, for example, Standard Definitions of Metropolitan Areas, a useful 
category for educational surveys. 

7. Processing and Interpreting the Data * Include In the basic design of the 
survey plan; 

a. procedures for editing and coding (methods proposed, codes, records, 
and provisions for evaluating coding procedures) 
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b. tabulation plans, illustrated by dummy tables and describing how to show 

"unknowns" and refusals (e.g., whether distributed or shown 
separately) 

c. processing plans (whether manual or mechanical, with provision for con- 

trolling the errors of processing) 

d. outline for the analysis and interpretation of results. 

8. Allowance for Pre-tests and Follow-ups. Allow for pre-tests of the 
feasibility of the plan, and for follow-ups necessary to increase the rate of 
response and the accuracy of the survey results. (Recommendations for pre- 
tests and follow-ups are given below.) 

9. Comparison with Data from Other Sources. Plan to compare the data with 
data from independent sources. Although such comparisons cannot be used to 
validate the survey results, they are often useful in Interpretating them, 

10. Proposed Calendar. Determine approximate dates for all steps in the sur- 
vey to ensure orderly completion of the project: beginning preliminary design 
work, beginning and completion of pre-test, beginning field work or mailing 
questionnaires, completion of editing and coding, completion of tabulations, 
completion of preliminary report, completion of final report (including 
evaluation of the variances and biases of the survey results). 

11 . cost Estimates , Estimate the anticipated total cost of the survey, in- 
cluding: 

a. preliminary and exploratory work 

b. development and printing of questionnaires, definitions, and instructions 

c. manuals and other instructions for the conduct of the survey 

d. collection (mailing or field costs, including interviewing, travel, and super* 

vision) 

e. editing and coding 

f. tabulation 

g. analysis 

h. preparation of final report 
I, overhead and other costs. 

The questionnaire or report form and accompanying instructions should follow 
the general principles set forth In ‘‘Standards for the Design of Report Forms” 
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(Bureau of the Budget, August 1945 — to be reissued as one of the Exhibits to 
Circular No. A-48), It gives recommendations for the form, the layout, and the 
printing. 

Pre-tests. The feasibility of most survey plans should be tested in advance. 
Pre-test i should be conducted to answer certain explicit questions: 

a relative effectiveness and cost of alternative questionnaires, instructions, 
and operating procedures 

b. acceptability and intelligibility of the questions by the respondents 
o, possible misunderstandings of questions and procedure by the inter- 
viewers 

d, clarity and applicability of definitions and classifications; reference dates 

e, completeness of questions for correct coding and interpretation 

f, defects in the forpis, maps, lists, instructions, etc, 

g, estimates of strata means and variances 

h, response rates' 

Without proper statistical design, the pre-test may fail to answer these 
questions. It /should be kept as simple as possible to elicit clearcut ans* ars. 
This criterion, considering the specific objectives and the cost and time 
elements/should determine: the choice of classes or localities where the pre- 
test is to be conducted; the approximate size of the pre-test (number of inter- 
views); the sampling plan within the classes or localities se- 
lected; the maps, lists, or directories to be used; the pre-test calendar, including 
start date and date when tabulations are to be made available; the records to be 
kept; the tabulations required; and the analyses to be made. 

The tabulations should be limited to those that answer the specific questions on 
the pre-test, and should generally not include additional tabulations for 
research purposes or for preliminary results for which the pre-test data may not 
be valid. Enough time should be allowed for analyzing the results or the pre- 
test, in case they are needed in modifying the plans of the main survey. 

Follow-ups. In most surveys, response to the first request for information, 
whether by mail or in person, is insufficient for final estimates; a follow-up is 
needed. In planning the number and timing of follow-ups, the response to the 
initial inquiry and the precision required should be estimated. The method of 
collection used in successive follow-ups—whether mail, call-back interview, or 
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other means-should be determined by the response rates that can be an- 
ticipated from alternative methods, considering their costs. 

Analysis of successive follow-ups and of Information on the characteristics of 
those not responding initially is generally needed to prepare final estimates 
from the survey. Information on characteristics of those unable or unwilling to 
answer may be useful in interpreting the data. 

Partial-Coverage Surveys. In partial-coverage surveys, the mailing or interview 
list should be carefully defined and fully described. The first step is to define 
the type of sample design-i.e„ whether a probability sample, a case study of 
type situations, complete enumeration of some previously developed list, a cut- 
off" (inclusion of all cases beyond or below a specified criterion), or some other 
plan. 

Selection of the method depends on the purpose of the survey. Probability sam- 
ples are needed where unknown biases of selection and estimation will be 
hazardous and where the precision of final results must be known. Use of the 
cut-off point often produces the data needed at minimum cost and burden. Sur- 
veys using cut-off points will be most efficient if designed with the aid of a cost 
function and the theory of probability to achieve a minimum mean square error. 

Once the method of partial coverage has been defined, the sample design 
should be described, and the universe defined, along with the "frame" or list of 
units from which the sample may be selected, the basis for grouping these 
units, the techniques of selecting particular units in each group in the sample, 
and the methods for summarizing the information collected and for drawing in- 
ferences about what has been revealed by the survey 

The description of the sample design might be checked against the outline 
below. It applies directly to probability samples, but many of its items are 
relevant to other designs as well. The terminology used in the outline is con- 
sistent with “The Preparation of Sampling Survey Reports" (United Nations 
Statistical Papers, Series C, No. 1 (Revised), 15 February 1950). 

1. sampling units: the primary units (description and number in universe) and 
the sampling units to be used at the second and higher stages (description and 
number in universe) 
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2, lists, maps, directories, and other aids to define the sampling units; con- 
struction of lists in the field 

3, criteria of stratification 

4, cost of function 

5. the formula for the variance 

6. the optimum allocation of the sample 

7. method of drawing tne sample at each stage, including the ratio sampled, 
by stratum and by stage, and the techniques of identifying the individual sample 
respondents (e.g., by use of random numbers, or by systematic selection from a 
random start) 

8, form of estimate proposed, for each variate of major importance: 

a. formula for the estimate; source of benchmark data if employed; weights, 

derivation 

b. universe parameters (proportions, variances, correlations, and their effect 

on the sample design) 

c. numerical evaluation of any bias in the formula of estimation 

d. formula for the expected variance 

e. numerical approximation to the variance 

9. statement regarding the proposed control and measurement of biases: 

a, methods proposed for reducing non-response 

b. revisits to measure failures and defects arising from under-coverage and 

over-coverage; misunderstanding the questionnaire and instructions; 
interviewers’ biases; incorrect definition and classification; various 
kinds of error in reporting 

In large-scale statistical surveys made by personal enumeration, one must 
develop well organized procedures to achieve good performance and maximum 
consistency. Establishing procedures for the conduct and supervision of field 
work should include: 
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1, methods of selecting, training, and supervising the enumerators 

2, specifications for training fleid workers (hours of training, methods and 
devices for training, whether payment Is made for training sessions) 

3, organization for supervising the field work. Including qualifications and ex- 
perience of the supervisors, method and frequency of communication between 
central and field offices 

4, basis of payment to enumerators (piece-work, hourly, etc.) and scale of 
wages and salary 

5, plans for measuring differences among enumerators 

6, plans for measuring major types of bias, whether arising from the 
enumerators, the questionnaire design, timing, or respondents 

7 , anticipated rate of response; differential rate by type of respondent 

S 5 specification of the method(s) used to deal with non-response— the num- 
ber of call-backs required and how to make them; how to correct for varying 
probabilities of individuals being available; subsampling of non-respondents 

9. provision for central control in large field studies, and establishment of 
procedures to ensure uniform and consistent rulings on matters not covered by 
instructions. 



Instructions should be developed for all phases of the survey to explain the 
basic definitions and procedures in detail. Depending on the magnitude and 
type of survey, materials should include: 

1. specifications for selecting and training enumerators, field supervisors, 
editors, coders, analysts, et al. 

2. manual of instructions for field workers, including procedures for listing 
and selecting the sample, training kits, etc. 

3. manuals of instructions for editors, coders, machine operators, et ah 
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Large-seals statistical surveys require systematic plans for reporting progress 
on each phase to determine whether the allowances of time and funds are 
being met and to permit any necessary modifications in the calendar. These 
reports should also note particular problems encountered, to help evaluate final 
results and plan other surveys. Items to be included in the periodic reports are: 

1. progress of the work, both in the field and in the office 

2. description of special difficulties or costs encountered in the field; 
proposed remedies 

3. description of special difficulties encountered in the offices; proposed 
remedies 

A. description of difficulties encountered with the definitions 

5. identification of special biases encountered (non-response, failure to un- 
derstand the questions, failure of the interviewers to understand the definitions 
and sampling procedures, etc.). 

At the conclusion of every survey, a cost analysis should be made, covering all 
the Items listed. The final costs should be compared with the advance estimates 
and summarized for future use. 



Good statistical practice for the conduct of a survey includes a careful presen- 
tation of the results, following the “Standards for the Publication of Statistical 
Data.” These standards, issued in 1957, were developed by the Bureau of the 
Budget in cooperation with the major statistical agencies of the government to 
reduce possible misunderstanding or misinterpretation of statistical data. 
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RESEARCH AT THE 

COAST COMMUNITY COLLEGE DISTRICT 



RICHARD W, BRIGHTMAN 
Director, Research and Planning 
Coast Community College District 



This paper on the organization of research activities in the Coast Community 
College District describes two basic types undertaken in the past two years, 

ADMINISTRATIVE ORGANIZATION OF THE DISTRICT 

The two colleges in the district, Golden West in Huntington Beach and Orange 
Coast in Costa Mesa, together enroll over 27,000 day and evening students. 
Each college is administered by a president, operates autonomously, and differs 
markedly from the other in administrative philosophy and practice They are 
served by a centralized district administration, both geographically and ad- 
ministratively separate from both of them. Most research activities are conduc- 
ted within the organization of the district administration. This is not to say that 
the campuses conduct no research of their own. 

Golden West, for example, maintains a Research Advisory Committee, con- 
sisting of administrators and faculty menbers, whose purpose is to conduct cer- 
tain research projects and to assist faculty members in individual research en- 
deavors. Despite these localized activities, most research takes place at the 
district level as part of a district administrative unit known as the Office of 
Educational Development. 

The Office of Educational Development is headed by a Vice Chancellor for 
Educational Development, who reports directly to the Chancellor for the district. 
Research is only one of several activities undertaken by this office. They in- 
clude aggressive efforts at seeking funds from government agencies such as 
the California Community Colleges, the California State Department of 
Education, and the U.S. Office of Education. The office is also responsible for a 
wide variety of developmental activities, Including in-service training programs 
for faculty members and a program permitting faculty members with develop- 
mental projects to submit them for funding out of a special budget appropriated 
by the Board of Trustees, a feature deserving additional comment at this point. 
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It is known as the Faculty Fellowship Program and in the last two years has 
provided over $100,000 to those with developmental projects to pursue. Linking 
activity with research is an important feature of our organization, as will be 
shown. 



As for financing, the district does not establish a specific research budget. In- 
stead, the total operational budget for the Office of Educational Development 
contains provision for all district-level research. Some research financing 
comes from sources outside the district, e.g., U.S, Office of Education and the 
Chancellor's Office of California Community Colleges, As an example, the 
recent two-year Project Follow-Through has been completely financed by the 
California Community Colleges and has provided our district with a research 
data file particularly useful In conducting follow-up studies and in designing 
other techniques for evaluating specific instructional programs. We also try to 
make provision in all proposals submitted to federal and state agencies for 
funds to evaluate the project. A division chairman leadership conference con- 
ducted in conjunction with the League for Innovation in the Community 
College, for example, contained a budget entry for research to evaluate the 
conference. Similarly, a grant from the Department of Justice to develop com- 
puter-assisted simulation materials for the Law Enforcement Program at Golden 
West provided for substantial research to evaluate the learning system it 
developed. 



Many community colleges closely associate che functions of research and 
hustling money from government sources. This association can be an advan- 
tage to the researcher because he can write in procedures for evaluating his 
projects and thereby acquire the wherewithal to conduct his research activities. 
It is pointless to conduct developmental activity without providing for 
procedures to ©valuta the results. 

RESEARCH ACTIVITIES 

The district’s research activities fall into two basic categories: first, institutional 
research, which produces descriptive information about the district and the 
students in it; and second, evaluative research, which concentrates on 
assessing how much different learning systems vary for better or worse from 
others in measurable student performance and, more recently, in relative costs. 
As one might suspect, the latter is considerably more difficult. We shall examine 
the easier one first. 
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institutional Research. Institutional research is the total collection of activities 
that provide information about the college district and its students and faculty 
to those who can use it. 

institutional research falls into two broad categories. First, it produces numbers 
that describe the district’s financial resources, expenditures, physical facilities, 
student populations, etc. These numbers appear in a looseloaf notebook main- 
tained for some 70 administrators, faculty members, and student leaders. En- 
titled Profile , the notebook has seven sections: Physical Facilities, Budgetary 
and Financial Information, The District Community, Enrollment Statistics and 
Student Characteristics, Faculty and Staff, Programs and Curricula, and Com- 
munity Colleges, The information in each secticn is up-dated as current data 
are received. Sources include the Orange County Progress Report prepared by 
the Orange County Board of Supervisors, the Directory of the American 
Association of Junior Colleges, financial and other reports prepared for the 
district, including five- and ten-year plans for physical plant and curriculum 
development, and statistical publications of the U.3. Office of Education. 

The Profile provides summary data on the district and both colleges. It supplies 
information not otherwise readily available to the faculty, administration, and 
■■ students, but which is often of interest to them, particularly when discussing the 

college with outsiders. It Is designed as a permanent reference book and is up- 
dated each semester or whenever necessary. Each individual on the mailing list 
for it receives the new information to replace the old in the notebook. 

The second type of institutional research describes student body performance 
and activities, and issues reports with such titles as “A Comparative Statistical 
Report of Grade Distribution and Unduplicated Registration Count in the Coast 
Community College District/* “Scholastic Standings of Students on the Dean’s 
List/* and “Facts Regarding the Graduates of Golden West College and Orange 
Coast College, Class of 1971." Their formidable titles alone give one an idea of 
what the reports are about. 

Evaluative Research. The district is more concerned with evaluative research 
than with institutional researen. The former tries to assess the relative effec- 
tiveness of different Instructional techniques. This means comparing the results 
of innovative learning systems with conventional teaching practices. Such com- 
parison comas in three varieties. In the first, a comparison of student perfor- 
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rnance under two instructional strategies uses student grades as a criterion for 
measuring changes in performance and is known as the “normative’’ measure 
of evaluation. By and large, educators give this most attention today, A second 
technique draws on student opinion about whether or net one instructional 
strategy is more or less effective than another. In the third, native instructional 
plans are evaluated by comparing how well students meet specific educational 
objectives as measured by well validated examinations. This last is the most dif- 
ficult evaluation to conduct and is known as the “summative” method (4), 

Despite all the literature on innovative projects In journals like Telecom - 
munications, Media , and Instructional Technology , relatively little is reported on 
evaluating the relative effectiveness of any one particular innovative technique 
compared with any other or with conventional instruction, Moreover, what 
evidence is available suggests that it really makes no difference. A report by the 
Center for Advanced Study of Educational Administration had this to say: 

We are able to state decisively that no particular method of college Instruc- 
tion is measurably to be preferred over another when evaluated by student 
examination performances. We may also conclude that replication of the 91 
studies examined in detail in this survey would not produce conclusions 
different from ours. Any future research on comparative teaching methods 
at the college level must move in new directions (3). 

Despite this gloomy initial evidence, it is hoped that different instructional 
strategies win' yield different educational results depending on the subject mat- 
ter and on the students studying it. 

One limitation of the study quoted above was dealing with evaluating students 
In a normative fashion, i.e., using grades as performance criteria as opposed to 
evaluating them summatively in meeting educational objectives. It seems that 
meaningful instructional evaluation must be conducted according to the 
specific educational objectives being sought by the educational system unde 
examination. In addition, some rigorous effort must be devoted to selecting 
control and experimental groups and to pre- and post-testing the students in 
the programs. Without such rigor, no effective evaluation can take place. Unfor- 
tunately, it is very difficult to achieve. By and large, teachers are unwilling to 
have their students treated as guinea pigs and they typically react negatively to 
the whole idea of evaluatic i efforts conducted In a rigorous, quantifiable man- 
ner, “After all,” many argue, “the things I teach and the people I try to teach it 
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to cannot be measured in quantified terms. How can you evaluate the quality of 
my effort by using numbers?*’ 

The difficulties in evaluating Instructional effectiveness are four in number. 
First, as mentioned already, teacher unwillingness to use their students as 
guinea pigs is the most troublesome problem. Second, making comparisons 
that are truly fair and selecting experimental and control groups that are truly 
comparable is most difficult. Third, establishing criteria of success or of 
evaluation is also most difficult, Finally, the validity of measuring instruments 
for educational programs is equally difficult to establish. All four problems com- 
bine in a total situation that makes the researcher’s job most formidable. 

Even in the face of these obstacles, several efforts to evaluate the instructional 
programs have been made. They Include each of the three basic forms of 
evaluation. One, for example, is a normative study of the relative effectiveness 
of two different procedures in handling remedial English. Starting with the fall 
1968-69, remedial English at Orange Coast was taught to students requiring it 
as an integral part of their English 1A Freshman Composition. It was a change 
from previous years when students needing remedial work In grammar* syntax, 
and other matters completed a remedial English course before enrolling in 
Freshman Composition. To assess the difference in performance, if any, bet- 
ween the students who studied Freshman Composition before and after the 
change, grade and attrition rate measures for English 1A students were 
gathered over a four-year period. These data established, with statistical com 
fidence at the 10% level, that students performed “better” under the new 
procedure than under the old. “Better,” in this case, meant that their grade- 
point averages were higher and their attrition rates lower (2). 

Another study, of the summative variety, compared performance on a written 
examination between a control and an experimental group. It tested how well 
the subjects had mastered certain concepts of computer operation. The ex- 
perimental group studied them with the use of the computer=assisted simulation 
exercises; the control group studied them in a conventional classroom environ- 
ment. In the study, statistical number-pushing did not show mean performance 
scores between the groups to be significant. 

Six procedures were followed in this study: 

1. Gomput@r=assisted instruction segments were designed to prepare students 
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to meet certain specified educational objectives, all of which were articulated in 
behavioristic terms* 

2. The segments were tested in two ways. They were used by students who 
were not to participate in the formal evaluation study and reviewed by a number 
of data-processing instructors from community colleges throughout the West 
Both students and instructors found them an enjoyable way to learn computer 

operations, They offered suggestions that were incorporated into the final 
learning materials. 

3. Control and experimental groups were selected from the 28 students com- 
prising the population of the study. Selection was based on SCAT II scores and 
on a questionnaire (administered in class) designed to assess how much 
previous training and experience each individual had. Students were assigned 
to either the control or the experimental group initially on their responses to the 
questionnaire. Next, the mean SCAT scores for the two groups were compared 
but no statistically significant difference was found. 

4. Instruction for the control group followed conventional classroom 

procedures; the experimental group used the computer-assisted simulation 
exercises. 



5, All 28 students were given an examination that consisted only of those 
items that tested directly for the established educational objectives. 

6. The examination scores of the two groups, when compared, showed no 
significant difference between their mean examination scores. 

Other efforts at summative evaluation of instructional programs include com- 
paring the relative effectiveness of two different learning systems used for a 
selected portion of law enforcement training at Golden West, Also, Initial steps 
have been taken to evaluate how much video-tape presentation in Cultural An- 
ropo ogy improves student performance when compared with that of students 
who do not view the tape. The district has not undertaken as many summative 
evaluation projects as it would like, but the few done have been highly infor- 

Informal evaluation, based on student opinion of classroom procedures and of 
courses m general, is typically performed by faculty members on their own. 
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Most often, those who want to evaluate how much students favor or disfavor 
their courses and procedures do not consult the District Research Office. ln= 
stead, they either design their own evaluation forms and procedures or use 
those available to them from the Dean of Instruction. The Research Office 
avoids trying to work into these efforts, being sensitive to the fact that faculty 
prefer to keep the results to themselves. Observation, however, leads to the 
belief that, when students are asked about the quality of an instructional 
program or course of instruction, most of them tend to bias their answers 
positively despite assurances that their answers wiii not affect their grades. 
Some faculty members have reported the comparative results of using student 
evaluation questionnaires on different types of learning systems. In most cases, 
it seems that the new instructional system outperforms the conventional system 
in the eyes of the student. 

Considerably more detail on all three types of evaluation activity conducted at 
the Coast Community Coilege District was published in a recent work entitled 
Strategics for Change. It is available from ERIC Document Reproduction 
Service, P.O. Drawer O, Bethesda, Maryland 20014, The cost is 65 cents in 
microfiche or $6.58 in hard copy (160 pages). Its order number is ED 051 806. 
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INSTITUTIONAL RESEARCH AT 
LOS ANGELES CITY COLLEGE 



BEN K. GOLD 
Director of Research 
Los Angeles City College 



INTRODUCTION 

One of the nation’s oldest and largest junior colleges, 43-year=old Los Angeles 
City College, serves a heterogeneous population of 20,000 students— all com- 
muters- - on its downtown campus. 

What effect does the college have on these people? Like almost every other 
educational institution asking itself this question, LACC has been hard pressed 
to come up with the answer. The first step was to establish the LACC Research 
Office to assist in the effort. 



Research was originally assigned to the Counseling Center, where it remained 
until 1966 when the Research Office was established as a separate entity. 

ORGANIZATION AND FUNCTION OF THE RESEARCH OFFICE 

The office’s general purpose and specific responsibilities were agreed on as 

follows: 

1. General purposes 

a. to provide objective and current evidence how well the c liege is 
meeting its stated objectives 

b. to furnish information to administration and faculty so that policy 
decisions and implementation shall be based on current and reliable fac- 
tual data. 

2 , Specific responsibilities 

a. to explore areas in need of research and make appropriate recommen- 
dations for conducting it 
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b. to stimulate, coordinate, and act as consultant for Gampus-wide in- 
stitutional research efforts 

c, to maintain a library of research materials, including on- and off-campus 
studies, both for reference and as a background for new studies, and to 
maintain a continuous inventory of Los Angeles City College research 
studies 



d. to recommend regular procedures for gathering and storing desired infor- 
mation 

e. to recommend procedures for disseminating the information to ap- 
propriate individuals and groups 

f- to conduct research studies as advised by the President of the Research 
Advisory Committee 

g* to cooperate with other colleges, the California Junior College 
Association, and other agencies in providing and receiving information 
and in conducting inter=instltutional research. 

The Office Director received part-time assistance from the Counseling Center 
secretarial staff and occasional student employees, and his office was placed in 
a staff position on the college organization chart, directly responsible to the 
president* , 

A 12-member faculty Research Advisory Committee was appointed by f h e 
college president with the following functions: 

1 . to determine areas where research is needed and make recommendations 
regarding institutional research 

2 . to recommend procedures for the dissemination of research findings 

3. to serve as a resource to the whole college for research design and 
procedures 



'The position has been shifted recently, under a general administrative reorganization, 
and is now part of the Office of College Development, Functions and duties remain the 
same. 
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4. to encourage faculty members to propose and participate in research 
projects, 

IMPLEMENTATION 

Twelve to 20 studies relevant to various aspects of the college program are 
produced each year, and may be requestor by any segment of the campus com- 
munity, The following annotated list of studies produced in 1970-71 illustrates 
the scope of both topics and Initiators. 

70-9 Persistence of LAGG Students Entering in Fall 1987 

A six-semester persistence study of a random sample of fall 1967 entrants, 
analyzed by sex, high school background, aptitude scores, initiated by the 
Research Office to provide baseline date (25 pp.) 

70-10 Grading Patterns at LAGG, 1955-1969 

A study requested by the Dean of Instruction to provide background data 
for considering changes in grading policies (24pp.) 

70-n The Fail 1970 Applicant Who Failed to Register: Who and Why? 

A study requested of all district colleges by district administration to in- 
vestigate the allegation that certain application and admission procedures 
"Turn away" students (14pp.) 

70-12 Academic Performance of Financial Aid Recipients, 1969-71 

A description of students receiving various kinds of financial aid and com- 
parison of their academic performance with the student body as a whole. 
This study elaborated on information required by funding agencies (21pp.) 

LAGG Faculty Time Allocation Survey 

7i-i A study requested by the College Committee on Evaluation and Standards 
to provide background data prior to recommendations on faculty 
evaluation procedures. It consisted of structured questionnaire interviews 
conducted by committee members (12pp.) 
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71-2 The Fall 1970 Guidance Examination 

Annual study of entering student performance levels on the College 
Guidance Examination, analyzed by sex, age, high school background, 
previous college (if any), and date of examination, and compared with 
past performances (17pp.) 

71-3 Academic Performance Oi .-ACC Transfers to the University of California, 
1969-70 

Annual analysis of first-semester performance of UC transfers, comparing 
performances in the various colleges and departments and with previous 
years (12pp.) 

71-4 Some Characteristics and Opinions of LACC Entering Students, Fall 1970: 
A Report on Responses to tha ACE 1970 Student Information Form 




Entering students completed the American Council on Education 1970 
Student Information Form, A subjective analysis of the ACE computer 
printout was made in this study. Requested by the Faculty Advisory Com- 
mittee (19pp.) 

71-5 Academic Performance of LACC Transfers to California State College at 
Los Angeles, 1969-70 

Annual study of first-semester performance at the college attracting the 
largest number of LACC transfers (over 800 per year) (8pp.) 

71-6 Survey of LACC Faculty Professional Development Activities 

A study requested by the college president to provide background infor- 
mation for possible policy changes relating to faculty teaching schedules 
(9pp‘) 

71=7 The Spring 1971 Psych 9 Program: Student Characteristics and Attitudes 

An evaluation of a new federally funded program using group guidance 
techniques to enhance “disadvantaged" students’ self-image and peer 
relationships. Requested by the Dean of College Development (30pp.) 



- 44 




39 



71-8 Some Characteristics of Withdrawing Students, Fall 1970 

Analysis of exit interviews of withdrawing students with counselors, 
probing reasons for withdrawal. Initiated by the Research Office after 
discussions with counselors (11pp.) 

Copies of complete studies are sent to campus and district administrators, 
members of the Research Advisory Committee, other faculty members directly 
concerned with the study, and to the ERIC Clearinghouse for Junior Colleges. 
One-page abstracts are prepared and distributed to all faculty members. 

Obviously, studies are not restricted by type. Some are purely descriptive; 
others are experimental (or at least quasi-experi mental, generally using designs 
no more complex than a simple experimental-control group comparison); still 
others may be correlational cr predictive. Most are arranged in five sections: 
purpose, procedures, findings, conclusions, and recommendations. Any 
statistically sophisticated language is limited to the “findings” section. 

The secant function of the Research Office is to maintain a “research library,” 
consisting o two collections — hard copy and microfiche. The hard-copy collec- 
tion, for the most part, contains ephemeral or fugitive materials: college 
research studies, journal reprints, testing company bulletins, papers by 
graduate students in junior college education classes, YSOE reports, Research 
Laboratory papers, and relevant junior college material. These items, currently 
500 in number, are cataloged by a simple ten-category classification scheme. 
The microfiche collection consists of all documents processed through the 
ERIC clearinghouse at UCLA and copies of Research in Education. Faculty and 
staff are invited o us€: the library at their convenience or, with ample notice, to 
request a search on a particular topic. Lists of new additions are distributed oc- 
casionally and, when a report of general faculty interest is received, a one-page 
abstract is prepared. 



The Research Office also provides a clearinghouse for questionnaires that 
reach the campus. Requests for information or opinion are channeled routinely 
to the Research Office, which provides the information or forwards the request 
to the appropriate department for reply. Along with this activity the Research 
Office, in cooperation with the Data Center, is developing a file of information 
most often requested. 
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STRENGTHS AND SHORTCOMINGS 

From a highly biased point of view, let us examine the strengths and 
weaknesses of the LACC institutional research endeavors. 

On the credit side we think we have made an impact on decision making. For 
example, remedial and developmental programs have been initiated and revised 
in accordance with studies evaluating needs and program effectiveness; 
student surveys based on random sampling designs have helped validate or 
reject claims of student support for certain issues; in-depth analysis of transfer 
performance has led to correction of curriculum and course inadequacies- per- 
sistence and academic performance studies analyzed by subgroups have 
helped to quiet extravagant claims of prejudice; a survey of faculty attitudes 
during an attempted student strike presented administrators with guidelines for 
future actions; studies of entrance examination performance and procedures 
have led to their modification. 

On the debit side we have a long way to gc to fulfill the objective of 
stimulating, coordinating, and acting as consultant for campus-wide in- 
stitutional research efforts." The Research Office produces about 90% of all 
campus studies. Administrative reorganizations, with possibilities of instruc- 
tional released time, will, we hope, improve the research output of other 
segments of the campus community. A closely related shortcoming is the small 
number of studies specifically on instructional outcomes. Obviously, with an of- 
fice working on a budget of less than a half per cent of the college’s operating 
budget, not everything can be done, and we still need continuing revisions of 



RECOMMENDATIONS 

Assuming that some who read this article are about to begin an institutional 
research office, we offer these guidelines from our experience: 

1. strive to make the office image that of a service agency to the colleqe 
(especially the faculty), not a bureaucracy 

2. concentrate your efforts on campus problems and issues, not on the 
“nature of the universe” 

3= set up data-collection procedures with specific purposes in mind— not 
just collection of data for its own sake 
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4. provide precise information and specific recommendations to the campus 
decision makers, but don't expect them all to be adopted. Remember that 
all decision makers are bound by their own constraints ana that some 
studies, e.g., measuring aspects of the college environment, will be direc- 
ted at understanding the nature of the college. They will have long=range 
rather than immediate use, 

5. select an Advisory Committee, representing as many campus areas as 
possible, as a sounding board for ideas 

6. be prepared to provide background material for funding proposals and 
accreditation teams, but make it an outgrowth of other studies, not a goal 
in itself 

7. consult with college committees as much as possible 

8. get as much help as you can in compiling and analyzing data — but never 
neglect a thorough study of the data yourself 

9. make your reports readable, using graphic aids where possible. Remem- 
ber that your readers have not lived with the data as you have 

10. don't be ‘‘set in concrete;’’ adapt to changing institutional needs 

11. exchange ideas with researchers at other colleges in your area, both in- 
formally and through an organization if possible 

12. read “The Nature and Role of Institutional Research ... Memo to a 
College or University,” a statement prepared for the Association for In- 
stitutional Research by Jo© L. Saupe and James R. Montgomery, Novem- 
ber 1970. 
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RESEARCH AND FACULTY INVOLVEMENT 
IN A JUNIOR COLLEGE 



ANN BROMLEY 
Director of Research 
Santa Fe Junior College 

Research in the community junior colleges has only recently become a formal 
activity. Periodic surveys have been conducted on the extent and role of in- 
stitutional research (2;3;4;5;6). 

Tangible evidence of this growing importance is the increase in the number of 
junior collage consortia being formed and of their representatives who attend 
the national conferences of the major research associations. During the past six 
years, the increase in attendance has been from less than five to over sixty par- 
ticipants. 

Although research in the junior college is receiving increased emphasis, it still 
has extremely limited staff and funds. Several of the surveys cited above 
inquired about the status and size of the budgets for research activities in the 
community junior college. The results were not particularly consistent, but they 
showed that, if a research office received from 3 to 5% of the college budget, it 
enjoyed extremely favorable funding. In many instances, funds were as low as 
0,5% of the total budget. Pieper (3) reported that two-yuar colleges with a 
research office spend an average of G.83% of the college budget on this func- 
tion and that over two=thirds of the research budget is for salaries. 

Even with these limitations, a viable program of institutional research can be 
developed if the researcher uses the talents of the faculty and the administrative 
staff. 

Critical Factors . Two factors are essential if faculty-administrator^student in- 
volvement is to be a positive experience: 



1, the philosophical environment of the institution toward research and the 
support by the other administrators are key factors; both are necessary to 
engage faculty and students in cooperative or college research activities 
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2, as community junior colleges have always prided themselves on being 
primarily teaching institutions, the image of the research office must be 
one of service, cooperation, and availability for assistance to faculty 
and/or students. 



Other factors may be strategic, but these two are fundamental, not only for at= 
trading faculty and students to research activities, but also for implementing 
some of the research findings in the college. 

Ways and Means. These five methods of involving faculty and staff in research 
and of educating „iem to its value in their classrooms and in the college are not 
all-inclusive, merely suggestive, but they have proven worthwhile and effective: 

1. communication techniques 

2. the use of a Research Advisory Committee 

3. faculty sponsors for college-endorsed research 

4. involvement in inter-institutional research and consortia 

5. participation in individual research and college committees. 

Communication Techniques. Each research office has a system of com- 
municating with members of the administration, faculty, and staff, and has effec- 
tive internal communication. It is undoubtedly aware that, as new staff members 
join the college, they too must be informed of the services and assistance that 
research can give them. 
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In 1969, the ERIC Clearinghouse for Junior Colleges developed a Topical Paper 
designed to stimulate research in the junior cc Hoge. It presents a model for in- 
structors, administrators, and researchers who wish to study the effect of their 
efforts. It begins: f ‘A junior college English instructor thinks his more mature 
evening students are performing better than his day students. A philosophy 
professor feels that reprimanding students about performance on an 
examination does more harm than good. A political science instructor calls on 
the library staff to help her increase student use of current history 
materials. . .. What do all these teachers have in common? All are interested in 
student learning and all are on the verge of formulating explanations that can 
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be tested for accuracy" (1:1). This readable Paper, “Is It Really a Better 
Technique? , may serve as a basic research primer for junior college faculty 
and staff who are somewhat unsophisticated in research techniques and 
research design. It furnishes them with a simple way to discover if their sup- 
positions are correct. 

One way of communicating would be to send each new staff member a copy of 
this Topical Paper early in the term, with a letter from the research officer in- 
viting him to scan it. It would bring to his attention that the college has a re- 
searcher who is willing to assist him, and that research may not be as for- 
midable as he had thought. 

Each year that this device has been used, at least *wo or three staff members 
who received the Paper in September have come to the office to discuss 
questions about their instructional process, and are frequently ready to examine 
and evaluate their own teaching-learning situation. Sometimes during the in- 
service training sessions for new faculty and staff, the Office of Research is 
asked to address them. The Topical Paper may be distributed then if it has not 
already been sent through the campus mail. 

A discussion of communication techniques should not overlook internal college 
news media besides the local and student newspapers. A good personal 
woiking relationship with the editor of the student newspaper and with the 
college public relations director is helpful in securing releases that involve 
research and are related to students or faculty. These two influential people are 
usually receptive to short write-ups on faculty members who are trying 
something different in their classrooms and more than willing to release an ar- 
ticle on the characteristics of the student body for the fall term. They are also 
helpful in releasing information on any national research projects in which the 
collage may be participating, and for which the students and faculty may con- 
stitute the sample population. 

Some junior colleges mail new faculty members a packet containing a college 
catalog, faculty and student handbooks, etc., before their in-service and/or 
orientation period. Before actual classes start, the staff is likely to read the 
material more carefully than at any other time. As initial impressions are impor- 
tant, it is therefore suggested that the information on the Office of Research, its 
services, and its relationship to the academic program be presented in both the 
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faculty handbook and the catalog In such a way that its service aspects are em- 
phasized. 

A most effective communication technique is for the Office of Research to serve 
as a clearinghouse, sharing reports, professional news items, and material from 
Junior Coliege Research Reviews . The research officer must be aware of the 
college’s activities so that he can disseminate information on them to the in- 
dividuals on campus with similar interests. For example, as information on com- 
puter-assisted instruction is received by the research officer, it should be for- 
warded to the interested faculty or staff member. He will generally reciprocate, 
keeping the research officer informed of his own activities and seeking 
assistance when he needs it. 

Most recommendations on the responsibilities and functions of the research oF 
fioer Include the maintenance of a central file system of the college’s research 
activities. Simp!y collecting and maintaining the files, however, is performing 
only about one-fourth of a valuable service — the information should be 
disseminated. 

One technique that has proven effective is a periodic formal publication on 
studies ranging from classroom activities to college-wide research, from short 
statements on different instructional approaches to dissertation abstracts, from 
subjective observations to intricate experimental designs, and from updated 
reports on the previous year’s document to projects initiated immediately 
preceding the publication. Santa Fe has published three such collections, the 
first in 1969. The most recent publication included two- to four-page abstracts 
and contained 20 instructional reports; six dissertation abstracts; five reports on 
inter-institutional projects, eleven on college research, and eight on college- 
endorsed projects. It is informative; it stimulates faculty and staff to try different 
approaches and to report them; and it provides personal satisfaction to its 
authors. It is distributed throughout the college, to the ERIC Clearinghouse for 
Junior Colleges, and nationwide (to those who request copies, if available). 

Research Advisory Committee. A Research Advisory Committee of faculty, staff, 
and students has considerable value. Students should be included on it for 
several good reasons. They make useful contributions to the development of 
any research project, they often help in its review, they may assist in cianfying 
the purposes and the appropriate populations of the projects, and frequently 
they constitute the study sample. 
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The composition of the committee is crucial to its effective functioning. For it, 
the research officer should attempt to nominate or select individuals who 
represent the college community. He should serve as chairman, as group 
leader, and be constantly aware that the committee cam 



1. assist the research officer in establishing priorities 

2. suggest possible areas of inquiry 

3. react to ideas of the researcher and others on possible studies 

4. serve as a communication channel for the research office, the faculty, and 
the students 

5 S analyze and constructively review the various research activit as of the 
college whether they are carried out by the research office, a faculty mem- 
bar, or a visiting graduate student. 



One responsibility of the Committee should be to establish ad hoc committees 
of other faculty members, students, and personnel to serve on a particular 
project. This committee would develop the project, identify the needed data, 
and implement the study. The research officer could serve as chairman of the 
ad hoc committee, or simply as a consultant and data source, The Research Ad- 
visory Committee can also help evaluate the research program of the college. 

Faculty as Sponsors of Research. As junior colleges increase and as training 
programs at various universities attempt to use them as laboratories for their 
graduate students, the junior colleges find themselves— students, faculty, and 
staff— under examination. This scrutiny is directly related to the proximity of the 
junior college and the university with a junior college training program. At 
Santa Fe, requests by graduate students to conduct research projects at the 
college are directed to the Research Advisory Committee for review and ap- 
proval. The Committee then requests that a Santa Fe faculty or staff member 
serve as liaison for the graduate student, make appropriate arrangements within 
the coiiege fo facilitate the project, and coordinate the college phase with the 
university professor directing the research. The faculty member thus beco .es 

involved in it, interested in its results, and aware of its implications for the 
college. 
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Involvement In Inter-institutional Research and Consortia. Within recant years, 
various consortia and research councils have grown up on the junior college 
scene, including the League for Innovation in the Community College, thg 
Florida Inter-Institutional Research Council, and G.T Participation has many 
advantages, not the least of which Is the in-service training It affords the 
research officer. 

Research councils frequently undertake projects that involve faculty and staff, A 
researcher is well advised to invite faculty to accompany him to the council 
meetings when their particular fields are discussed and to workshops that are 
related to faculty problems and needs. 

Recent workshops involving Sante Fe faculty or administrators have been on 
divisional chairman leadership, projection analysis, the Program Planning 
Budget System, and the junior college presidents discussing the role and func- 
tions of research officers and how to involve them in the decision-making 
process. Such activities establish a better working relationship between the 
research and academic areas of the college and provide the expertise for the 
academician to participate in any project from its beginning. 

In summary, organized research in the junior college is a fairly new 
phenomenon. It is an educative and in-service training activity for many of the 
faculty and staff who come to the junior college from high school or university 
environments. For the research to have meaning, the research officer must ex- 
tend his services to the entire junior college program. To offset a limited budget 
and staff, faculty support is an excellent resource for developing the research 
potential. Five ways (communication techniques, the use of a research advisory 
committee, faculty sponsors for college-endorsed research, Involvement in in- 
ter-institutional research and consortia, and participation in individual research 
and college committees) were suggested for research to make a greater con- 
tribution to the junior college. 

BIBLIOGRAPHY 

1. Connor, Aiken. "Is It Really a Better Technique?" Los Angeles: ERIC 
Clearinghouse for Junior Colleges. Topical Paper no. 6, March 1969. 



2. Johnson, B. Lamar. "Institutional Research in Western Junior Colleges." 
Junior College Journal, March 1962. 

O 

ERIC 






S3 



49 



3, Pieper, W,D,, Jr, Financial Support for Institutional Research, 1969-1970. 

Berkeley, Calif: Association for Institutional Research, 1971. 

4, Roueche, John, and Boggs, John. Junior College institutional Research: 

The State of the Art. Monograph no. 2. Washington, D.C.: American 
Association of Junior Colleges, 1968, 

5, Swanson, Herbert Leland. "An Investigation of Institutional Research in 

Junior Colleges of the United States," Unpublished doctoral disser- 
tation, University of California, Los Angeles, 1965. 

6, Van Istendal, Theodore. "Community College Institutional Research,” 

Paper presented at Association for Institutional Research Forum 
Chicago, Illinois, May 1969, 




54 



INSTITUTIONAL RESEARCH IN 
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HENRY MOUGHAMIAN 
Coordinator, Instructional Services 
City Colleges of Chicago 



CURRENT STATUS 

If Institutional research is “any endeavor that leads to the improvement of the 
institution / 1 It Is not surprising to find such a variety of research practices. Its 
problems are as varied as the educational backgrounds and experiences of the 
Individuals who direct it. With such recent attention to the training and nurture 
of researchers, present directors of institutional research are often people 
trained In psychology, sociology, statistics, data processing, etc. Considering 
their different backgrounds and the magnitude of the problems they face (from 
cost analysis to theoretically-oriented research), the confused state of the 
research they eonduc* should not be alarming? If it cannot be clarified with all 
its current financial and moral support, it will remain in disarray. To structure its 
institutional research, an institution must "know itself.” 



.Therefore, if an institution begins its research program on “student needs” and 
Ihe supporting educational services, It has taken the necessary and most impor- 
tant first step. Continuous institutional self-evaluation and provement can 
result from well-designed research, e.g., in counseling, library, or testing ser- 
vices, not only in relation to student needs, but aiso to finance. 

Few junior colleges have successfully systematized a research program. They 
have lacked the three components necessary for success, namely, trained 
researchers; adequate financing, supporting services, and staff; and a plan of 
action. Even though the need for Institutional research is increasingly 
recognized, its development still lags. The current emphasis on accountability 
may stimulate the needed research. 

ORGANIZATION AND DEVELOPMENT 

The direct " of institutional research must be part of the chief administrator’s 
staff, so that he can see both long-range and immediate needs, but his func- 
tions must not be so broad and time-consuming that, however well-qualified, he 
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becomes Ineffectual. Adequate staff and supporting services, such as data 
processing, are Indispensable to him. Without them, his office is severely han= 
dicapped. One person, with limited resources, cannot perform all the functions 
of an institutional researcher, for much of his time would be spent on both inter- 
nal and external reports and questionnaires. To expect a well-organized and ef- 
fective program from a one man office is sheer folly. 

Even though data collection, in itself, is not institutional research, the research 
office can help organize a system to satisfy many of these Inquiries, The 
following guidelines could be helpful: 

(1) the position of Director of Institutional Research must be a staff rather 
than a fine position 





(2) philosophy, policy, and direction for the research should be developed as 
soon as possible and given adequate organizational strength, staffing, 
and budget 

(3) an advisory committee should be formed to interest faculty and staff in 
planning and implementing the research program 

(4) broad participation by faculty and staff in doing actual research should be 
encouraged; it is neither desirable nor practical for the research office It- 
self to attempt all institutional studies, but rather, it should provide leader- 
ship and support 

( L , provision for us© of the findings should be assured; they should be widely 
disseminated within the system, as well as to those directly concerned 

(6) for a viable institutional research program, in-service training of faculty 
and staff on its nature and Its importance to decision making is recom- 
mended. 

CURRENT RESEARCH AT THE CITY COLLEGES OF CHICAGO 
The City Colleges of Chicago was founded in 191 1 1 but an office of institutional 
research was not established until 1966. Since CCC is a multi-college in- 
stitution, each college has its own research director. Some of the research ac= 
tlvitles conducted in recent years are; 
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(1) Student Characteristics 

As in most institutions, descriptive studies of student characteristics are con- 
ducted on a regular basis. With the growth of the occupational and technical 
program, the inputs and outputs of these programs are beginning to receive 
more attention. The Illinois Junior College Board, in cooperation with the state 
community colleges and American College Testing P ogram, recently did a 
comparative study of students in the technical-occupational and in the transfer 
programs. Even though the community colleges are supposed to emphasize 
counseling, few, if any, measuring instruments have proved useful in student 
guidance. Recently, both the College Entrance Examination Board and 
American College Testing Program have developed “guidance instruments'’ 
specifically for community college students. They are, in essence, mini-data 
systems Including demographic information, interest scores, and ability scales. 
This information is essential for guiding the students into beneficial programs, 

(2) Student Success 

Indices of student success are studied on a regular basis. They include course 
retention, academic performance, number of graduates, and number and perfor- 
mance of transfer students. A recent statewide study of the public and private 
two-year and four-year institutions (“Perforrnance.of Transfer Students Within 
Illinois Institutions of Higher Education”) identified approximately 30,000 tran- 
sfer students. Their performance was studied for one yean One finding was that 
as many students were transferring from senior institutions to junior colleges 
as from junior to senior colleges. Also, nearly half the students at the junior 
colleges were admitted on probation. This type of study can help in determining 
the success of transfer students and in statewide planning and articulation. 




(3) Students' Attitudes , Values , and Perceptions 

R, Edmund Dolan, using the Institutional Self-Study Survey, recently studied 
how students in different programs (Vocational-Technical, College Transfer, 
and General Studies) assessed (a) the quality of instruction, (b) the value of 
student services, and (c) specific college policies, practices, and facilities. He 
also sought to determine if various success rate measures were related to the 
students’ assessment of the college, A student survey like this can be helpful in 
an institution’s self-evaluation and improvement Similar feedback by faculty 
and staff is also most helpful. Another study by Ernest Jaski of the City Collages 
of Chicago investigated the impact of the junior college environment on student 
values and retention. 
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(4) Learning 

The topic of learning has been examined for many years, but except for a few 
general principles, the research has net been fruitful. The teaching-learning 
process is so complex that it is difficult to isolate and study the relative in- 
fluence of such variables as the abilities and characteristics of the learner, the 
method of instruction, learning experiences, class size, materials, and 
evaluation techniques. When the heterogeneity of community college students’ 
abilities, aspirations, and interests is added to the already complex learning 
situation, questions will inevitably arise about the adequacy of the programs. 

Research reiated to learning has been scarce in the community college. Since 
junior colleges have many low-ability students, both programs and methods 
deserve extensive study. A survey conducted by the College Entrance 
Examination Board entitled ‘‘Developmental Programs in Midwestern Com- 
munity Colleges” (Higher Education Surveys Report No. 4) by Richard I. Ferrin 
reviews current activities, placement procedures, staffing practices, and 
evaluation techniques. 

Television instruction has been used by the City Colleges of Chicago since 
1956. Thanks in part to the extensive preparation necessary (development of 
study guides and course objective learning materials, and evaluation devices), 

it has proven to be an effective instructional tool, especially for highly motivated 
adults. 



The use of the computer in instruction has increased in the past few years. Even 
though many doubt that it is an effective tool, research has yielded little data on 
its impact on learning. National Science Foundation support of regional net- 
works has helped develop software and inspired enthusiasm for computer ap- 
plications. Problems encountered by the CCC in using one of these networks 
were the lack of faculty training in computer technology and poorly developed 
computer problem-solving programs in the academic disciplines. The high cost 

of computer use, especially interactive systems, makes its value in the learninq 
process questionable. 

A current project by Emmett Jones at Olive-Harvey College in Chicago is the 
type of research that holds much promise for the community college. Using Ben 
Bloom’s ‘‘learning for mastery” concept, Jones is working with faculty members 
to individualize the instructional program for their students. The mastery con- 
cept ranges from relating sequential learning units to characteristics of the 
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learner to evaluating the total learning process. Other research projects, in 
various disciplines, are needed to evaluate its effectiveness. 

MEASURING INSTRUMENTS 

The construction and validation of achievement examinations can be most 
valuable in measuring the effectiveness of instruction. Once objectives are 
stated in behavioral terms, their attainment can be measured not only by tests 
but by other devices as well. Expressing achievement in qualitative terror can 
be useful both in evaluating growth and in furnishing local norms for com- 
parative purposes. ,s there can be no accurate prediction without reliable 
criterion measures, the implication for institutional research is clear. At CCC, 
an extensive computerized item analysis program gives faculty the opportunity 
to develop effective examinations in their own subject fields. These analyses 
provide valuable data not only on attainment of course objectives but also on 
the discrimination value or the individual test items. The use of poor evaluation 
principles has been a major shortcoming in community colleges and in 
education in general. 



The current emphasis on credit by examination gives us the chance to allow for 
individual differences through the preparation of both self-study materials and 
suitable examinations. Examinations prepared by the major testing companies, 
especially the technical and occupational, are limited in scope. A recent study 
at the CCC contrasted characteristics of successful and unsuccessful students 
in its credit-by-examination program according to ability, sex, highest level of 
educational aspiration, previous college work, enrollment status, and age. This 
type of study and follow-up studies of the successful students can be helpful in 
counseling students toward their educational objectives. 

TRENDS 

Student participation in research activities has been primarily as respondent to 
demographic and planning surveys. Today, students’ views are being sought by 
more and more institutions on policy making, evaluation of the instructional 
program, student personnel services, and facilities. To a more limited extent, 
the same is true of faculty participation. Faculty support for research must be 
increased. Most institutions need a “research center," i.e„ a center where 

faculty may obtain help in developing experimental designs and evaluation 
devices. 
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Not only does every community college need a comprehensive research 
program, but it also needs more inter-institutional cooperative research 
projects, from descriptive studies to controlled experiments. Recent years have 
shown a trend to such broad-scaled research. Kenneth Anderson and others at 
the University of Kansas completed several studies of community college 
students as a part of their Master Planning Commission report. Banning Han- 
scorn of the Minnesota State Junior College System recently followed up the 
transfer students of all eighteen Minnesota junior colleges. A similar study was 
done in Illinois. 

With campus-wide cooperation, free access to a computer, a solid data-base 
(and a little luck), the researcher will have more time for qualitative analysis of 
his institution. 
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INSTITUTIONAL RESEARCH AT 
HARCUM JUNIOR COLLEGE 



BORIS BLA1, JR. 

Director of Research 
Harcum Junior College 

“An institution of higher learning that stops growing, improving, implemen- 
ting, and evaluating is a danger to both society and itself. What right does 
an institution have to exist when it no longer gives its best in attempting to 
cope with the increasing complexity of the society it serves?” (7:1). 

A college must continuously analyze its approaches, methods, and programs to 
insure the best possible education for its students. Under established 
management policy, Harcum* systematically gathers information through self- 
evaluation studies and uses the data in management, faculty, and staff 
decisions to modify (improve) existing practices. 

In this paper, institutional research is defined as “self-study by a college 
designed to improve the institution” (1). The definition applies not to a single 
unit such as faculty, students, or administration, but to all dimensions of the 
college. It is a “shared” activity; faculty, administration, and staff actively par- 
ticipate, as shown by the following: 

1. Faculty participate in policy deliberations primarily through six standing 
committees, academic division meetings, the Faculty Assembly, and the College 
Council. In 1970-71, of the 72 policy-and-procedure actions recommended by 
these groups, 21 were on research inquiries by faculty members. 

2. At the request of various faculty committees, the Office of Research con- 
ducted 14 additional research investigations, furnishing reports to facilitate ap- 
propriate recommendations. 

3. The Research Office believes “that successful linkage between educational 

Harcum is a fully accredited, private, independent junior college for woman located in 
Bryn Mawr, Pennsylvania. Its full-time day division enrolls between 600 and 700 students; a 
part-time evening and summer division enrolls between 200 and 300. 
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research and practice is achieved when user and (researcher) interact 
collab natively, stimulating each other’s problem-solving behavmr” (4). 

The Director of Research reports directly to the president and is responsible 
for: 

1. the aevelopment and conduct of the centralized college institutional 
research function, and for stimulating, heiping, and coordinating the studies of 
others to avoid duplication 

2. all activities necessary for on-going evaluation cf how well the college 
achieves its self-imposed objectives 

3. cooperation with other college personnel on theoretical ("pure”) research 
to expand basic knowledge in the complex teaching/learning process 

4. conducting, at the direction of 'the president, studies on administration, ad- 
missions and enrollment, curriculum, faculty, finance, goals and objectives, in- 
struction, plant, public relations and development, and students 

5. encouraging participation of faculty, administration, and staff personnel, in 
both the planning and conduct of research 

6. providing personnel with definitive data (descriptive and/or quantitative) to 
help them make decisions. 

To help insure orderly and comprehensive institutional research, a program 
ckeck-list is essential, for it determines the scope of topics for inquiry. The 
detailed "Areas of Institutional Research” cited by Swanson (8) have been a 
valuable guide at Harcum for both the cyclical research efforts (for long-range 
trend analysis and planning) and spur-of-the-moment inquiries (for immediate 
decision making). 

Instant solutions to immediate problems are not part of the process, although 
both facuity and administration must watch both long-range and day-to-day 
operations. Combined, the two functions (iong-range and immediate) 
have comprised our total researcn effort for several years. Undoubtedly ad- 
ditions can be made to Swanson’s list (8), but it has served us admirably. We 
recommend it to our readers. 
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Although the volume of research activities is not necessarily a valid criterion of 
their quality, it reveals both their extent and variety. During 1969-70, the 

heseaiui Office prepared 93 institutional research reports; for 1970-71, the total 
was 74. 



Since 1963, the Office of Research has published 45 articles in professional 
journals; since 1967, it has prepared some 229 IRR’s (Institutional Research 
Reports) and RM’s (Research Memorandums). The latter are brief notes on 
alumnae achievements; curricula, programs, etc.; faculty comments; student 
achievement, characteristics, comments, and evaluation; freshman applicants 
•and the like. 



The various studies conducted fall conveniently into three broad categories 
(numbers in parentheses show the percentage of academic year 1970-71 
studies in the particular category): 

1. Survey Research (38%): the collection of data by questionnaires, analysis, 
and records or interviews of either samples or whole populations. 

2. Experimental Research (50%): experimental manipulation of selected (in- 
dependent) variables, often using experimental and control groups to note what 
effects, if any, occurred in the criterion (dependent) variables. 

3. Historical Research (12%); answering certain questions by examining 
college records for valid information and by careful statistical treatment of it. 

We all know that on occasion even the carefully developed statistics can be 
misleading. Like a ventriloquist’s dummy, the figures can be made to say 
whatever their manipulator wants to say. Fortunately, our statistics have been 
more valid than the above quotation would suggest. 

Our principal techniques for data collection have been: (1) before-and-after 
evaluation, (2) check-lists, (3) correlational studies, (4) interviews— structured 
and open-ended, (5) inventories— personality and skills, (6) observation 
schedules, (7) parallel-group surveys, (8) questionnaires/opinionaires, (S) rating 
and ranking scales, (10) statistical treatment of records data, and (11) tests. 

Many of the techniques for junior college self-evaluation may be locally 
developed; others are better for regional or group action; still others are best 
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used on a nationwide basis. The self-appraisal needs and techniques we follow 
include: 

1. construction or adaptation of an academic test for adults 

2. interviews to estimate both the student’s experience in a field and his likely 
skill level 

3. a standard system to keep current with community employment needs and a 
pattern to describe students accurately to prospective employers 

4. a technique to describe prospective transfer students to four-year schools 

5. tests to measure academic characteristics of student groups for assessing 
programs and instruction 

6. measurement of non-cognitive characteristics of students to assess the 
collegiate environment or :i press,” effects of home environment on personal- 
social adjustments, shifting vaiues, etc. 

7. a clearinghouse for employment information and maintenance of applicant 
qualifications records 

8. joint study of characteristics of successful transfer students and of in- 
stitutional characteristics of the receiving four-year school to help two-year 
school counselors with future transferees 

9. transfer agreements with four-year schools on acceptable levels of 
academic achievement and test performance 

1C. cooperative studies of college and high school curricula to facilitate con- 
tinuity in the sequence of their academic programs 

11. criteria for study of junior college administrative practices 

12. local validation of hig school records and test data as predictors of suc- 
cess, for both academic a; d vocational students 
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13. appropriate measures of vocational interest, skills, and aptitudes to assist 
counselors 

14. follow-up studies of former students for curricular revisions 

1 5. a student personnel program statement of goals and evaluation criteria (6). 

Swanson (8) has noted that “few junior colleges with institutional research 
programs have developed any way of evaluating the effectiveness of their ef- 
forts, and most of those only in a subjective, nonscientific way." In spite of the 
“non scientific” nature of the following observations, they have yielded useful 
results at Harcum. 

A report by an evaluation team from the Middle States Commission on In- 
stitutions of Higher Education noted that because a// at Harcum were convinced 
of the worth of self-evaluation, it will undoubtedly be continued. The team also 
reported that every institution determines how well it is achieving its objec- 
tives— though never to its complete satisfaction- and that Harcum’s self-study 
efforts indicate considerable success. 

During the spring of 1971, some 90% of the faculty and staff responded to an 
anonymous questionnaire on their opinions of the Institutional Research 
Reports, which present findings to all who want or need them. 

Virtually all respondents (94%) rated IRR’s as being of “some” to “exceptional” 
value, most (52%) clustering at the mid-point rating on a five-level scale. (No 
claim is laid to equal-step-gapping or other sophisticated measurement for this 
simple, face-valid rating scale.) The “interest" rating for IRR’s follows the same 
general evaluation pattern. 

During the academic year 1970-71, student interest in institutional research was 
shown by the fact that, without any college pressure, direct or implied, each 
issue of the student newspaper carried at least one item reporting on it. 
Students also constantly expressed interest in results of the various question- 
naires that solicited their views, comments, and valuations. Numerous informal 
student contacts (such as dropping by to “rap” and sample the ever-available 
candy) indicate considerable awareness of the Office of Research. 

On the basis of these multi-evaluations and informal feedback from the Board of 
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Trustees, it has been the consensus of college management that institutional 
research efforts have been of practical benefit to the college. In substance, they 
agree with Marsee (6): “Education has became big business, institutional 
research is becoming increasingly important. Today enlightened educational 
leaders use the most advanced management techniques. In this setting, in- 
stitutional research is a must.” 

Since Harcum decision makers continue to seek improved methods of college 
operation, the rationaie for institutional research is crystal-clear and com- 
pelling. Under progressive, innovative management, institutional research has 
moved into the mainstream of college operations. By becoming directly in- 
volved in filling information gaps, it is relevant to all institutional problems and 
operations: (1) research inquiry, (2) conclusions, (3) recommendations, (4) 
decisions, and (5) planning activity— in short, the necessary steps prior to 
management decisions on new or modified practices. 

Although it may be self-evident, it is pertinent to note that the essential 
management function of self-evaluation is either an impossibility or 
meaningless “window-dressing.” The endorsement of the research program by 
executive management must be total. Management has supported Harcum’s in- 
stitutional research efforts with freedom of inquiry, financial support, and 
fcllow-up of research findings. 

Since Harcum is a private school, without public tax support, the financing of 
its institutional research is wholly determined by college executive 
management, with no sustaining public money. (This is available only under 
grants received for the conduct of specific projects.) On occasion, when the 
project warrants, special funds are furnished by the appropriate college depart- 
ment (e.g., academic division or administrative office). 

As Cottrell / T) has noted, “IR (institutional research) must be adequately finan- 
ced, a recommended amount, according to recognized leaders in junior college 
research, being 2 to 3 per cent of the college budget, or a minimum of $10,000 
to $25,000 for a small college on a yearly basis.” 

To this I would add a brief comment on staffing. Efficiently used, free time of 
personnel in other offices can he a valuable source of staffing support. For 
example, the telephone receptionist, in her spare time, can assemble completed 
reports; clerks in the mail room can address envelopes or dispatch question- 
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naires; and student help can free staff from the routine te!!/-work of research- 
generated data. 

Certain problems have arisen in disseminating the results (tangible products) of 
our research because of the diversified backgrounds of our various “publics.” 
Informing them most effectively has been a challenge in communications, for 
we seek to engender among them a positive attitude toward institutional 
research. Routinely, the reports are reviewed to see which particular study 
should go to whom and how best to send it. At present nine “publics” receive 
research information in nine forms. 

An effective program of institutional research should: 

1. Formulate a basic philosophy of institutional research, one that projects for 
a particular institution “the creative, dynamic spirit of curiosity.” This may be 
done through continuing and enlightened administrative leadership and 
through wide involvement of staff members. 

2. Provide specific and adequate financial support for institutional research. 
Someone once said, “A budget is the most important statement of the 
philosophy of a college.” The financial support of institutional research should 
not be left to chance. 

We at Flarcum have therefore concluded that the pragmatic answer to “why in- 
stitutional research?” is that it provides ways and means for identifying and 
analyzing the college’s problems. This is the first major step toward improving 
current programs and operations and planning i n + el ligen tly for the future (3). In 
short, institutional self-study is necessary to avoid the twin pitfalls of com- 
placency and misdirection. 

One final aphorism: Flaste makes waste. Most innovative programs in 
education, including institutional research programs, have been plagued by the 
tendency of both observers and participants to want immediate and visible 
results. We suffer from the habit, as former Commissioner of Education Flarold 
Flowe, put it, of pulling up the plant by its roots every few months to see if it is 
alive and growing! 
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OPERATIONAL NOTES FROM 
AN AMATEUR RESEARCHER 



RICHARD W. BRIGHTMAN 
Director, Research and Planning 
Coast Community College District 

I’ve come to recognize that education is replete with amateurs — teachers of 
business management have never managed a business, astronomy teachers 
have never been astronomers, music teachers do not play, and teachers of com- 
puter programming have never written a program for a real-life applicaton. This 
is piobably as it should be. Certainly, it is not only possible but perhaps 
desirable that teachers teach others to do what they themselves do not do. 

A number of graduate schools of education offer extensive programs in 
research arid evaluation. Alumni from these institutions doubtless fill research 
posts at community colleges, as a result, since many of us are amateurs in the 
research business, I speak from this perspective. I have found certain aspects 
of day-to-day research operations particularly noteworthy, and offer my obser- 
vations on them in the hope that others may be spared seme trouble. 

This paper, dealing only with operational matters, leaves discussion of the 
organization of research activities and of specific research projects to its com- 
panion piece, “Research at the Coast Community College District.” 

DATA GATHERING 

Of all the various activities of a researcher, I look upon data gathering as one of 
the most demanding. Many of the data we summarize and distribute about our 
student population and various district operations can be gathered by 
examining various official records such as budgets, grade reports, transcripts, 
and the like. Though tedious, this can yield information as accurate as the time 
and effort spent collecting it afford. The most demanding kind, however, in- 
volves soliciting information from students, ex-students, and other individuals 
on a personal basis. We have sent questionnaires to college presidents and 
chanceHors, to graduates, to withdrawing students, to dropouts, to faculty mem- 
bers, to employers, and to students while in class. We have employed the U.S. 
mails, telephone questionnaire techniques, and even faculty members to 
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distribute and gather questionnaires, both in class and at various local in- 
dustrial concerns. As a result, I have concluded that questionnaires (2), 
however distributed, are no way to gather accurate information. 

Most colleges gather basic data about individual students when they register 
and fill out applications for admission. This is not very promising as fat as the 
quality of information obtained. Consider, for example, the application-for- 
admission form, which is filled out by our students in one of two circumstances 
—either at recruiting sorties to local high schools or when the students come 
to the admissions office (or bring one they have received through the mail). 



As neither situation permits a thorough audit of the form, we have come to 
question whether the information given on it can be relied on at all. For exam- 
ple, one question asks for ethnic background. The application form gives a 
number of choices to be answered by putting an “X” in a box beside such items 
as “American Indian,” “American Negro,” “Caucasian or other White,” 
“Mexican American,” “Oriental,” and so on. This seems a fairly straightforward 
question, but analysis of how students respond suggests otherwise. From their 
answers, we had 537 American Indians attending our district during the spring 
semester, 1970-71. One correspondent remarked that ours must be the large 
single off-reservation group of American Indians in the United States. Among 
possible explanations for this obvious error are that “American Indian” is the 
first item to be checked and that many students identify with American Indians, 
even in personal dress and grooming. It doesn’t really matter why the number is 
wrong— we cannot use it. 



if we can’t discover such fundamental information as ethnic or racial 
background, we suspect that more esoteric information would be even more dif- 
ficult to gather. We ask applicants, for example, what their status was on 
leaving any previous college they attended. We also ask them if they are 
working for a degree, how many college units they have earned, and so forth. 
We also ask them for their address and telephone number. As with many junior 
colleges, many of our students drop out or withdraw after the first semester. A 
couple of years ago, we tried to follow up every student who left — first, by 
sending them a mail questionnaire, and then by telephoning those who did not 
return it. A surprising number of returned questionnaires, bearing “permanent” 
addresses, were marked “undeliverable” by the post office. Our telephone 
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procedures yielded similarly disturbing results. Over half of those we tried to 
call had given us numbers from which they had moved or had never had in the 
first place. 

When prospective students fill out an application form in absentia, there is no 
practical way to audit their answers or to check their address i telephone num- 
ber, family income, racial background, or previous college experience. ‘When 
they register for classes, however, one expects that the personal contact with 
officials in the process would influence them to improve the accuracy c* the in- 
formation they give. 

Typically, we gather little information from students when they register. 
Registering 1,500 or more students in one day leaves little time for questions. 
For a few semesters a nine-question survey, the size of an IBM punched card, 
was given to each student at registration. I asked about work hours pe week, 
college major, transfer plans, etc. Later, we revised the questionnaire ’ic tly, 
using the same questions, but changing the format for easier key pun hing. 
inadvertently, both forms of the questionnaire appeared in one of the 
registration lines at Orange Coast College. As a result, 687 students answered 
the nine questions at one point in the registration line and the same ones a few 
minutes later, the only difference being in the questionnaire form itself. 
Discovering this error, we compared all the questionnaires that had been an- 
swered twice, with startling results. Students had answered the questions dif- 
ferently, even though they were exactly the same ones they had answered only 
moments before. The percentage of students who gave different answers varied 
from one question to the next . One question was answered differently by only 
16% of the students; another, by 86% of them (3). Such results cast a shadow 
on the value of student data collected at registration. 

Students doubtless had important reasons to answer so inconsistently. One 
could be the format of the questionnaire, which has an important influence on 
the way it is answered. How can one determine, without personally interviewing 
students and verifying their answers, which format yields more nearly accurate 
answers? Another reason, perhaps even more important, is that students are 
mainly interested in enrolling in their necessary classes before they are filled. 
As a result, they do not look charitably upon answering questions before or af- 
ter signing for their classes. Furthermore, college registrars have little more, If 
any, charity toward research activities conducted during the process, for they 
want to move as many students as possible along as fast as possible. In sh 
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students aren’t concerned about answering accurately and administrators 
aren’t concerned about seeing that they do. 



If getting information from students under these conditions is difficult and 
yields inaccurate results, a more controlled situation should yield better infor- 
mation. One Such situation should be the classroom itself. For a while, we 
delivered two-item questionnaires to each student for every course he took. 
They were identified with the student name, the course name, and the instructor 
name. They were packaged by course and delivered to the instructors, who 
gave them to the students. The students answered the two questions and gave 
them back for return to the Research Office. Even in this large district, few 
faculty members reported that they found the data gathering prohibitively cum- 
bersome. On the other hand, judging from the reaction of their students, most 
thought it unlikely valuable information could be gathered in class. Many in- 
structors did not piay the game at all, for 238 of the total 1,685 classes did not 
return the questionnaires. In other words, 103 of the 716 faculty members did 
not return the questionnaires sent to them (4). 

We maintained the in-class questionnaire for three semesters. We were then 
convinced, from faculty member feedback, that students were not taking them 
seriously, even though they asked only (1) why the student was taking the class, 
and (2) if he thought the class would be valuable to his long-term career plans. 

We believe that, if tender loving care is lavished on selected faculty, in-class 
questionnaires can provide accurate and valuable information for evaluating 
specific segments of a college program. Our attempt, however, to gather infor- 
mation from all courses and all programs was a hopeless task, even for such a 
broad area as vocational or trade and technical education. The mam problems 
are the logistics of producing, distributing, and gathering back all the. question- 
naires and, even more important, convincing faculty and students that the infor- 
mation is important enough to make it worth their while. 

Not all uses of questionnaires, of course, are fruitless. Educational 
professionals provide good data, for they look on it as a professional respon- 
sibility to give information to those who are conducting surveys and working on 
other research projects. Faculty members and administrators in our district 
have cooperated on such surveys, but as our primary responsibility is gathering 
data to help evaluate and improve our own educational program, we look to the 
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student as a most important source of information. Questionnaires are not very 
effective in tapping this source. 

It would be unfair, of course, to write off questionnaires without offering an 
alternative. In the school year 1971-72, we plan to investigate two such alter- 
natives. The first is the patterned interview, in which a small sample of students 
(including graduates, withdrawn students, and enrolled students) will be per- 
sonally interviewed to gather specific information about experiences in college 
and how we might have made the experiences more valuable for them. We hope 
to use a similar technique to gather information from employers, parents, faculty 
members, and other clientele of our colleges. A second technique uses panels 
of students, faculty members, citizens, lay advisers, and the like. These panels 
have a variety of advertising and market research applications in industry. We 
can doubtless find a way to apply the technique to community college 
evaluation. 

DATA PROCESSING 

Most institutional researchers, amateur or professional, rely on centralized com- 
puter services if available. Lacking these, they use an adding machine or desk 
calculator and perhaps a clerk or two for basic tabulation tasks. Either way, 
data processing is an operational problem for the researcher, for even if a com- 
puter is available, he is likely to be frustrated in his efforts to use it. No com- 
puter center, with its other important duties, has time left over for new ap- 
plications in a research project. 

Heretofore, researchers have simply had to queue up with others and hope their 
job wi!! be done sooner or later. We have solved much of the problem, however, 
through the use of teleprocessing. Our office has a computer terminal giving us 
access to the Coast Community College District computer system, using a 
programming language known as APL (A Programming Language). We can 
thus circumvent certain difficulties and delays normally associated with using a 
largo centralized batch computer system. 

To help the frustrated researcher, I’ve tried to pass on this good news about 
teleprocessing. With such a system, most research applications, particularly the 
experimental or evaluative, can be accomplished most readily, though not the 
large reports with all their reproduction of statistical data. It works for such 
worthwhile research as assessing the effectiveness of alternate l^arnino 
systems. The type of terminal device we use can be made available to almost 
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any research function for surprisingly little money. The typewriter terminal and 
a device for transmitting and receiving via telephone iines costs less than $150 
a month. For about $12 per hour, high-powered computer systems can be made 
available through this terminal. This access to high-powered processing ability 
and mass data storage thus costs less than a good secretary. Learning to 
operate the system requires only three or four days' time. 

REPORT DISTRIBUTION 

Gathering data and transforming it into usable form assumes that it will be 
deliverf/d to those who can use it, both inside and outside the college. The 
problem is to determine who should and should riot receive which report. The 
simplest solution is to send a copy to ail who might have even the slightest in- 
terest. in it. With a large budget for paper and other supplies, this approach is at 
least economically feasible. It is vulnerable, however, to those who charge that 
the Research Office is only a paper mili, filling the school’s wastebaskets. The 
other extreme is hardly an answer either — sending reports to no one. This ap- 
proach would lead someone, if not the researcher himself, to conclude ^nt he 
is really not needed. 



Our distribution system falls between these two extremes. Reports on financial 
resources, expenditures, physical facilities, student population, etc. go to some 
70 administrate r s, faculty members, and student leaders. Those on procedural 
matters, producer periodically, are distributed to a special mailing list, which is 
regularly upda-ed. 

CHOOSING RESEARCH PROJECTS 

Some say that research projects should not be undertaken unless some college 
official is cleariy committed to use the information for decision-making pur- 
poses, helping him to choose between controversial alternatives. There is much 
to say both f r and against this approach. If followed, it would provide infor- 
mation only to those bd le to soecify 3xae:ly what they need. Only a few 
decision makers have go firm a grasp on what they need and we chould provide 
them with it if we can. For the most part, however, we can best help decision 
makers by having a wide variety of information available, and by helping them 
choose the information most helpful for their decisions. 

A second, though soecious, argument against this approach is that research ac- 
tivities, as they accumulate information about the organization, may bring to 
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light completely unanticipated facts. As a result, entirely new fields of inquiry 
and new categories of decision may bubble to the surface. 

• he approach we follow is that research projects be undertaken at the request 
of almost anyone in the district or on our own initiative. We investigated the ef- 
fects of new remedial English procedures on Freshman Composition students 
when asked to do so by the English Department. The chancellor asked us to 
examine the relative costs of auto-tutorial instruction in biology compared with 
conventional instruction. We also gather data on student performance in all 
classes solely to have it available for whoever might want it for whatever pur- 
pose. 

READING RESEARCH REPORTS 

The conscientious researcher will try to prevent his readers from misinter- 
preting his findings. It is easy to offer no interpretation or conclusions, as if 
yoi sr task weu ily to present the information, but researchers have a respon- 
sibility to try \o give meaning to all these figures. If gathering data is difficult, 
presenting it and drawing reasonable conclusions is equally challenging. 

It is as important for the researcher to show what he did not find as what he did. 
Extreme care must be taken to explain the significance (or lack of significance) 
of positive, negative, and inconclusive findings. Readers are prone to accept 
only those parts of a report that fit their prejudgments. 

CONCLUSION 

This amateur researcher has attempted to comment on what is and is not going 
on in research and on what will soon be required — referring, of course, to the 
current interest in accountability and the general trend toward mechanistic ap- 
proaches to evaluation. 

Accountability; the development of planning, programming, and budgeting 
systems; performance contracting; and many other topics promise make 
projects for the researcher, whether amateur or professional. Our few forays 
into comparative cost analysis of instructional programs (5) foretell a monumen- 
tal job. An accurate cost analysis of Z\ community college instructional program 
will require either the use of elaborate cost allocation formulas (6) or a com- 
plete restructuring of the college accounting system. 

These words of Robert Wolff may help the researcher maintain his perspective 
and balance: 
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Ironically, paradoxically, there are some human activities in which subjec- 
tivity is more efficient that objectivity, in which calculation kills and instinct 
inspires. Art and love are notoriously of such a nature. I believe that 
education is also (7). 
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